Research
Paper breakdowns
A breakdown is our technical reading of one influential paper: what problem it tackled, how the method works, what the authors report and where the evidence stops.
Numbers are the authors’ own reported results, quoted from the paper and checked against the source. We don’t re-run the experiments, and we say so.
Every breakdown covers
- TL;DR
- The problem
- The method
- What the paper reports
- Why it mattered
- Limitations
- Reading notes
Tagged “Scaling”
1 breakdown of 6
Chinchilla: how to split a compute budget between model size and training data
A technical reading of the 2022 paper that concluded large language models were undertrained — the three ways it estimated compute-optimal scaling, the Chinchilla-versus-Gopher test, and why later work revisits the recipe.
- Scaling
- Large language models
- Paper authors
- Hoffmann et al.
- Paper year
- 2022
- arXiv identifier
- arXiv 2203.15556
New breakdowns by email
Sign up for email updates on new paper breakdowns. No issue has been sent yet.