Research
Paper breakdowns
A breakdown is our technical reading of one influential paper: what problem it tackled, how the method works, what the authors report and where the evidence stops.
Numbers are the authors’ own reported results, quoted from the paper and checked against the source. We don’t re-run the experiments, and we say so.
Every breakdown covers
- TL;DR
- The problem
- The method
- What the paper reports
- Why it mattered
- Limitations
- Reading notes
Tagged “Architectures”
2 breakdowns of 6
Attention Is All You Need: the architecture behind modern language models
A technical reading of the 2017 paper that replaced recurrence with self-attention — what the Transformer does, why it trains faster, and what the paper did and did not show.
- Architectures
- Large language models
- Paper authors
- Vaswani et al.
- Paper year
- 2017
- arXiv identifier
- arXiv 1706.03762
Retrieval-Augmented Generation: pairing a language model with a searchable memory
A technical reading of the 2020 paper that introduced RAG models — how a dense retriever and a sequence-to-sequence generator are trained together, the difference between RAG-Sequence and RAG-Token, and what the results do and do not show.
- Retrieval (RAG)
- Large language models
- Architectures
- Paper authors
- Lewis et al.
- Paper year
- 2020
- arXiv identifier
- arXiv 2005.11401
New breakdowns by email
Sign up for email updates on new paper breakdowns. No issue has been sent yet.