Take one model and one competition problem. Sampling once, sampling 64 times and voting, or letting the model think for tens of thousands of tokens first can differ by more than ten points of accuracy and by tens of times in cost. Since 2024, "reasoning models" have turned this into a training target and a product setting. Extra inference compute pays off only under two conditions: the model has a fair chance of reaching the right answer, and something can pick the right sample out, or let the model correct itself within one chain. Without either one, thinking longer just costs more.
Draft, pending review. The site owner has not reviewed this note yet; facts, emphasis and wording may change. Check the primary sources linked in the text before relying on it.
What test-time compute means
Once training is done the weights stay fixed; any extra computation spent answering a question is test-time compute (also called inference-time compute). Generating one token costs roughly $2N$ floating-point operations ($N$ is the parameter count, attention not included), so extra compute is mostly extra generated tokens; Snell et al. (2024-08) count it as $2N$ times the number of inference tokens. There are four ways to spend it, and real systems often combine them:
| Way to spend | Where the extra compute goes | What it needs | Examples | Typical failure |
|---|---|---|---|---|
| Sample many, then pick | N independent answers to the same question | Answers that can be compared, or a verifier | Self-consistency voting, best-of-N | The right answer is not the most common one; the verifier accepts wrong answers |
| Think longer | Checking, backtracking and switching methods inside one chain | A model trained to use long chains | o1, DeepSeek-R1 | Overthinking easy problems, hopping between ideas on hard ones |
| Search over intermediate steps | Score each step, keep only promising prefixes | A process reward model or value estimate | Beam search, Tree of Thoughts | The search finds the scorer's blind spots |
| Generate, check, revise | A program or model checks, then the answer is revised | A checker, or a model that checks itself | Unit tests, Lean, proof verifiers | The specification being checked differs from the question you care about |
Two axes: parallel and sequential
Parallel means generating N independent answers at once and picking one by vote or verifier. As long as the server has room, latency barely grows with N while cost grows linearly. Sequential means thinking longer within one chain, or revising each version after looking at the previous one: later steps can use earlier findings, at the price of latency that grows with length.
On PaLM 2-S* and the MATH dataset, Snell et al. found that the effectiveness of each approach "critically varies depending on the difficulty of the prompt" (abstract), and that the two axes suit different problems: easy problems gain more from sequential revision, while on hard problems the best budget is a mix of both. Most systems today use both. DeepSeekMath-V2 (2025-11) starts each problem with 64 proofs, verifies each one 64 times, then revises the high scorers for up to 16 rounds; OpenAI's GPT-5 system card (2025-08) says gpt-5-thinking-pro in ChatGPT uses a "parallel test time compute" setting; Google DeepMind says the Gemini Deep Think that reached a gold-medal score at IMO 2025 explores and combines several candidate solutions at once. The last two did not publish the details.
This series uses one teaching example throughout: a math problem whose answer can be checked, where the model answers correctly with probability $p$ on a single try. Note 1 asks how many samples to draw and how to pick; note 2 asks how one chain gets trained to be longer and how its length is controlled; note 3 asks how to compare the numbers in release posts.
Timeline
| Date | Event | Organisation | In one line |
|---|---|---|---|
| 2021-10 | Training Verifiers | OpenAI | GSM8K; generate many answers, pick one with a trained verifier |
| 2022-03 | Self-consistency | Sample several reasoning paths, vote on the final answer | |
| 2023-05 | Let's Verify Step by Step | OpenAI | A process reward model picks answers better than an outcome reward model or voting; PRM800K released |
| 2024-07 | Large Language Monkeys | Stanford and others | Coverage grows roughly log-linearly with the number of samples |
| 2024-08 | Compute-optimal test-time scaling | UC Berkeley / Google DeepMind | Allocate test-time compute by difficulty; it does not trade 1:1 with more parameters |
| 2024-09-12 | o1-preview | OpenAI | Long chains trained with RL; self-reported smooth gains with both training compute and thinking time |
| 2024-12-20 | o3 on ARC-AGI-1 | ARC Prize / OpenAI | 6 samples 75.7%; 1024 samples (about 172x the compute) 87.5% |
| 2025-01-22 | DeepSeek-R1 (Nature version 2025-09-17) | DeepSeek | Open weights and report: RL with rule-based rewards alone makes answers longer and more accurate |
| 2025-02-24 | Claude 3.7 Sonnet extended thinking | Anthropic | One model can answer fast or think at length; the caller sets a token budget |
| 2025-04-16 | o3 and o4-mini | OpenAI | The model decides when to call tools during its reasoning |
| 2025-07-21 | Gemini Deep Think at the IMO | Google DeepMind | 35/42, graded by IMO coordinators against the student standard |
| 2025-08 | GPT-5 system card | OpenAI | A real-time router chooses between a fast model and a thinking model; the pro setting uses parallel test-time compute |
| 2025-11-27 | DeepSeekMath-V2 | DeepSeek | Train a proof verifier, use it as the reward, and scale verification at inference |
| 2026-02-05 | Claude Opus 4.6 | Anthropic | Adaptive thinking: the model decides whether and how long to think; the caller only sets effort |
| 2026-02-23 | Stops reporting SWE-bench Verified | OpenAI | Flawed tests, and frontier models have seen the problems and fixes |
| 2026-03-25 | ARC-AGI-3 | ARC Prize | Interactive environments; at launch humans 100%, frontier AI 0.51% |
| 2026-04-24 | DeepSeek V4 preview | DeepSeek | One model with thinking and non-thinking modes; low / high / max levels added on 2026-08-13 |
| September 2026 | Navier–Stokes proof | OpenAI | Self-reported: about 10,000 concurrent agents for about 88 hours, with a Lean formalisation; what it settles is disputed (Scientific American, 2026-09-21) |
What has settled and what is still moving
Each cell points to a dated public record. The "signals" column says what kind of result would change the picture; it is not a prediction that it will happen.
| Settled | Still moving | Signals to watch next |
|---|---|---|
| Sampling more raises coverage, but without an automatic verifier you cannot pick the right one: Brown et al. (2024-07) went from 15.9% with 1 sample to 56% with 250 on SWE-bench Lite (where tests decide correctness directly); voting and reward models stop improving after a few hundred samples | Better ways to pick: DeepConf (2025-08) filters reasoning traces by the model's own confidence and self-reports up to 84.7% fewer generated tokens than full parallel sampling on AIME 2025; DeepSeekMath-V2 (2025-11) trains a dedicated proof verifier | On tasks without reference answers, selection accuracy approaching coverage under independent grading |
| RL with verifiable rewards alone can train long chains, and length grows with training: DeepSeek-R1-Zero (2025-01 report, 2025-09 Nature version) | Whether RL widens the capability boundary or only makes existing solutions easier to sample: Yue et al. (2025-04, NeurIPS 2025) versus ProRL (2025-05) and Wen et al. (2025-06); the question-by-question analysis by Wu et al., updated 2026-10-14, still finds more shrinkage than expansion | Results across several model families, with reasoning checked, where pass@k at large $k$ also stays above the base model |
| The benefit of test-time compute depends on problem difficulty and does not trade 1:1 with pretraining: Snell et al. (2024-08) | Who decides how long to think: from 2026-02-05 Anthropic lets the model decide; as of October 2026, the OpenAI, Anthropic and Google APIs all use discrete effort levels as the main control; Fan et al. (2026-08) found that models do not allocate a budget shared across problems by difficulty or point value | Scores on shared-budget evaluations that reward allocating compute by difficulty |
| Written reasoning is not necessarily the computation the model relied on: Turpin et al. (2023-05); Chen et al. (2025-05) measured that Claude 3.7 Sonnet and DeepSeek R1 mention a hint they used only 25% and 39% of the time on average | How long chain-of-thought monitoring stays useful: Baker et al. (2025-03) found that putting a monitor into the reward, under strong optimisation, teaches the agent to hide its intent in the chain; researchers from many organisations jointly called for preserving monitorability in 2025-07 | Labs publishing monitorability measurements in system cards and stating whether training pressure is applied to the chain of thought |
| Old benchmarks are contaminated or saturated: MathArena (2025-05) found most models score 10–20 points higher on AIME 2024 than their AIME 2025 results predict; OpenAI stopped reporting SWE-bench Verified on 2026-02-23 | New rulers: ARC-AGI-3 (2026-03-25) launched with frontier AI at 0.51%; MathArena evaluates on newly released competitions as they happen | New benchmark results reported at equal compute and with confidence intervals |
How to read this series
It helps to read these first: decoding strategies (sampling parameters), rejection sampling and best-of-N, verifiable rewards, GRPO's group-relative signal and DAPO. Neighbouring notes: generate, verify, retrain shows the same verifier ceiling inside a training loop; choosing a model for an agent covers "who decides whether to think"; Looped Transformer covers computing more rounds in the hidden state without writing tokens; how reliable a judge is, see LLM-as-a-Judge.
- Sampling, voting, and verifiers: why voting only helps when the right answer is the most common one, how a verifier's error rate sets the ceiling, process reward models and search, and what Snell et al.'s compute-optimal result actually says
- How long chains of thought are trained: what R1-Zero did and why length grows, distillation, controlling how long to think, overthinking and underthinking, and whether the chain is faithful
- How to compare and read the numbers: comparing at equal compute, pass@1 versus pass@k versus maj@k, the "RL only sharpens" debate, contamination and saturation
- Review questions: interview questions and a self-check