Search
Search
Find paper breakdowns, models, benchmarks, AI tools, services and guides across the site.
5 results for “transformer”
Paper breakdowns
4 paper breakdowns- Attention Is All You Need: the architecture behind modern language modelsA technical reading of the 2017 paper that replaced recurrence with self-attention — what the Transformer does, why it trains faster, and what the paper did and did not show./research/attention-is-all-you-need
- Retrieval-Augmented Generation: pairing a language model with a searchable memoryA technical reading of the 2020 paper that introduced RAG models — how a dense retriever and a sequence-to-sequence generator are trained together, the difference between RAG-Sequence and RAG-Token, and what the results do and do not show./research/retrieval-augmented-generation
- LoRA: fine-tuning large models by training small low-rank updatesA technical reading of the 2021 paper that freezes pre-trained weights and learns low-rank update matrices instead — the maths, why it adds no inference latency, what the authors report against full fine-tuning, and what remains open./research/lora-low-rank-adaptation
- Chinchilla: how to split a compute budget between model size and training dataA technical reading of the 2022 paper that concluded large language models were undertrained — the three ways it estimated compute-optimal scaling, the Chinchilla-versus-Gopher test, and why later work revisits the recipe./research/chinchilla-compute-optimal-training