You don't need to read the directory from top to bottom. Start with the thing you're stuck on: the computation inside a model, how to train it, or how to tell whether it improved.
This page covers how to read the foundations and model notes. To practice coding, study training methods, or work on system design, the study map has an entry point for each.
Pick a route
| What you want to do | Where to start | A useful checkpoint |
|---|---|---|
| Learn language models systematically | Learning map → core mechanisms → implementations | Trace tokens to logits, identifying the input and output of each step |
| Diagnose a model that will not learn or only fits training data | Generalization → activations and initialization → optimizers | Tell implementation, optimization, and generalization problems apart, then design a small controlled comparison |
| Understand learning from images and text | Embeddings and similarity → CLIP → Visual language models | Distinguish pair scoring from answer generation, and trace both data and gradients |
| Read a new model report | Model-reading exercise → family notes | Identify the exact version, its changes, and the experiments supporting an explanation |
| Train or post-train models | Feedback to objectives → methods → evaluation | Explain what a label means, what the loss rewards, and how to catch side effects |
| Build memory, retrieval, or agents | Memory lifecycle → retrieval → agents | Trace the evidence used for a request and locate the stage responsible for a failure |
| Prepare for interviews | ML questions and implementations → unfamiliar chapters → run a small example | Explain an example, a limitation, and a test without looking at the notes |
Understand both the module and its connections
Treat this diagram as a checklist, not a prescribed architecture.
flowchart TD
A["Data and feedback: what was observed"] --> B["Objective: what should be learned"]
B --> C["Model and representation: how information is computed"]
C --> D["Retrieval / memory / tools: how the task gets done"]
D --> E["Evaluation: what works and what fails"]
E -. Check labels and coverage .-> A
E -. Check objectives and constraints .-> B
A dissatisfied user may reflect stale memory rather than a small model. A higher score may reflect an easier candidate set. Looking one stage upstream and downstream is often more useful than replacing the model immediately.
Read each chapter in three passes
What comes in, what computation happens, and who uses the output? Mark difficult formulas for later.
Check shapes, denominators, masks, and assumptions. Change one condition and predict the effect.
Pick an example, explain the design, and consider a different assumption. Return to the notes wherever you get stuck.
You don't need to memorize the wording. If you can state the assumptions, walk through the reasoning, and give an example of your own, that's already good.
Connect the chapters
- Foundations → model families: understand attention before comparing why designs such as GQA change it.
- Data → training → evaluation: behavioral labels such as clicks or likes usually only approximate the real goal. When you change the training signal, check whether the old metrics can still measure the improvement you want.
- Memory → retrieval → agents: not everything worth storing is worth retrieving; retrieved content must not become permission to act.
- Principles → practice → papers: test your understanding in a small example, examine constraints in public projects, then check the scope of the original evidence.
Want to add a note?
It doesn't have to start as a full tutorial. Begin with something you didn't understand, add an example that can be calculated or run, then explain the reasoning and limitations. Separate public evidence, your interpretation, and untested ideas.
Open an issue for missing topics, or read how to contribute. One clear note is worth more than a directory filled with outlines.