metadata
library_name: transformers
license: apache-2.0
datasets:
- lkevinzc/numia-1.5-qa-concatenated
base_model:
- HuggingFaceTB/FineMath-Llama-3B
Llama-3.2-3B-NuminaQA
Links
- 📜 Paper
- 💻 GitHub
- 🤗 Oat-Zero Collection
Introduction
This model serves as a 3B base in our minimalist R1-Zero recipe.
Training details:
- Base model: HuggingFaceTB/FineMath-Llama-3B
- Dataset: lkevinzc/numia-1.5-qa-concatenated
- Epochs: 2
- Learning rate: 1e-5
Citation
@article{liu2025understanding,
title={Understanding r1-zero-like training: A critical perspective},
author={Liu, Zichen and Chen, Changyu and Li, Wenjun and Qi, Penghui and Pang, Tianyu and Du, Chao and Lee, Wee Sun and Lin, Min},
journal={arXiv preprint arXiv:2503.20783},
year={2025}
}