Some problems need serial steps

"In which city is the university of A's manager?" One approach looks up the manager, then the university, then its city. Intermediate results can help, but this is not a computational lower bound: parallel algorithms, different representations, or existing knowledge may reduce serial work. Multi-digit addition is similar: the usual digit-by-digit carry is one algorithm, not the only way to add.

Within a forward pass, looping increases serial network depth, not necessarily correct reasoning steps. The illustration deliberately allows only one link per pass to show the budget change. Real attention can aggregate several positions at once; it is not constrained by this toy one-layer/one-hop rule.

Diagram Multi-hop: one more link per pass

This interactive figure needs JavaScript.

What theory and experiments say

  • Giannou et al. (2023) construct networks that execute programs. Representability does not establish that next-token training will discover those programs.
  • Yang et al. (2023) show parameter efficiency on particular in-context learning tasks, not a universal compression ratio for language tasks.
  • Saunshi et al. (2025) compare shared and independent depth and find parameter-efficient performance on tested reasoning tasks. First separate the budgets being compared:

$$D_{\text{shared}}=D_{\text{untied}}=kL,\qquad P_{\text{shared block}}=kP_{\text{layer}},\quad P_{\text{untied block}}=kLP_{\text{layer}}$$

This is parameter accounting at fixed width, excluding extra components, not an accuracy formula. The study also finds tradeoffs in perplexity and some memorization tasks. It does not establish that reasoning depends only on depth rather than parameters.

The looping mechanism: a k-layer block looped L times, written (k⊗L), is in effect a weight-shared deep model. Two baselines: (k⊗1) with the same distinct parameters, and (kL⊗1) with the same depth and L times the parameters.
From the paperThe looping mechanism: a k-layer block looped L times, written (k⊗L), is in effect a weight-shared deep model. Two baselines: (k⊗1) with the same distinct parameters, and (kL⊗1) with the same depth and L times the parameters.Source: Saunshi et al., 2025 · Looped Transformers · Figure 1 · CC BY 4.0 · resized and converted to WebP

How it relates to CoT

Chain-of-thought adds computation by generating intermediate tokens. With KV caching, each new position traverses the network without recomputing the whole prefix. Looping instead updates hidden states, without requiring an output token per loop. The two can coexist, and a visible CoT is not automatically a faithful account of the model's decision. Generating more tokens is the serial axis of test-time compute; for how it compares with the parallel axis of sampling many answers, see Reasoning models and test-time compute.

Saunshi et al. give an existence construction in which a looped model simulates fixed-length CoT, with added width, heads, and placeholder tokens. This does not make arbitrary pretrained weights equivalent to CoT when repeated, or guarantee that training finds the constructed parameters.

Where it stops

Extra computation can help use available information, but supplies no new external facts. If the manager's university is missing, more loops do not look it up; retrieval or additional input is needed. Ouro's synthetic knowledge-capacity result is a measurement, not a theorem for all models; see costs and limits.