Self-improvement: how to measure "AI speeding up AI"

You read "over 80% of a lab's merged code is written by AI" and "a model can complete tasks that take humans 12 hours". Which one shows AI R&D is accelerating? Neither, directly: the first is a code share, the second a task-level capability, and how much faster research as a whole goes is a third quantity. Ask first who measured it and in what unit. As of October 2026, externally checkable task-level measurements are still rising but have hit the limits of the measuring tools; numbers for overall R&D acceleration come almost entirely from the labs themselves, and every estimate that gives an overall progress multiplier is below 2x.

Sort the evidence into four kinds

• Task-level measurements: METR's time horizon, RE-Bench and the like. Others can check them, but they measure "how long a task can be completed", not research progress.
• Lab determinations against their own thresholds: the "crossed / not crossed" verdicts in system cards, mostly based on internal evaluations.
• Numbers labs self-report about themselves: code share, agent hours, staff surveys, slopes of capability indices.
• Statements and predictions: see the guide; not repeated here.

The 50% time horizon: definition and fitting

Kwa et al. at METR define it in their 2025-03-18 paper as the time humans typically take to complete tasks that AI models can complete with 50% success. For each model, fit against each task's human time $t$ (§3.1):

$$ P(\text{success}\mid t)=\sigma\big(\beta\,(\log h-\log t)\big),\qquad \sigma(u)=\frac{1}{1+e^{-u}} $$

$t$ is the geometric mean time of successful human baselines on the task; $h$ and $\beta$ are fitted. At $t=h$, $\sigma(0)=0.5$, so $h$ is the 50% horizon. Continuously scored tasks are turned into success or failure with a threshold chosen to represent human performance, and each task is weighted by the inverse square root of the size of its task family, so that large families do not dominate.

Work it once

A teaching example with made-up numbers: one model runs 10 times on each of four task groups.

Convert success rates to logits and the model becomes a straight line in $x$:

$$ \operatorname{logit}p=\ln\frac{p}{1-p}=\beta\,(\log h-\log t) $$

The four points are symmetric around $x=1$, so the least-squares line has slope about −0.74 (per doubling of human time) and crosses 0 at $x=1$, giving $h=2^1=2$ hours. Two hours is the geometric midpoint of 1 h and 4 h, not the arithmetic midpoint of 2.5 h, because the axis is log time. The same line gives the 80% horizon: $\operatorname{logit}0.8=1.39$, $x=1-1.39/0.74\approx-0.86$, about 0.55 h, or roughly 33 minutes. The 80% horizon is usually much shorter than the 50% one, as METR's data below also show. METR actually fits weighted maximum likelihood on individual runs; the logit line here is only for hand calculation. With four symmetric points both methods put the 50% crossing at 2 hours, while the slope and the 80% horizon differ somewhat.

When the reading lies beyond the tasks

Take a stronger model. The 15-minute group is still 9/10; the other three groups are:

In the last two rows every group succeeds more than half the time, so the 50% point can only be extrapolated along the line, beyond the longest task. One extra success in the longest group moves the estimate from about 34 hours to about 108. That is why METR warns that readings above 16 hours are unreliable; its 2026-03-20 methods note adds that none of the suite's tasks is over 30 hours long.

This figure needs JavaScript.

Trend numbers, and the premises behind them

METR's 2025-03 paper reports in its abstract that the frontier horizon has been "doubling approximately every seven months since 2019, though the trend may have accelerated in 2024"; its extrapolation to month-long software tasks within 5 years is explicitly conditional. METR's Time Horizon 1.1, released 2026-01-29, grew the suite from 170 to 228 tasks and the tasks of 8 hours or more from 14 to 31; the post-2023 doubling time is 131 days (165 under the old TH1), and since 2024 it is 89 days (109 under TH1). METR's current data file gives 128.7 days (95% CI 104.4–158.0) for 2023 onward, excluding models whose 50% point is above 16 hours.

Converting 131 days: a year is about $365/131\approx2.8$ doublings, so the horizon would multiply by about 6.9 per year. That is arithmetic on a past trend, not a forecast. Read such numbers together with four premises:

1. METR's Frontier Risk Report of 2026-05-19 (assessment window 2026-02-16 to 03-16, with participation from Anthropic, Google, Meta and OpenAI) says the most capable agents essentially saturated TH1.1, and that the most capable shared model's 50% point estimate was between 16 and 20 hours.
2. The same report: on tasks of 8 hours or more, at least 16% of successful runs were illegitimate on review. In METR's 2026-06-26 measurement of GPT-5.6 Sol, the horizon is about 11.3 hours (95% CI 5–40 hours) if cheating counts as failure, 71 hours if cheating attempts are discarded, and beyond 270 hours if cheating counts as success; METR says none of these is a robust measurement.
3. METR's 2026-03-20 methods note: correcting for noise in task-length estimates could reduce frontier 50% horizons by about 30%.
4. METR's time-horizons page has not been actively updated since 2026-09-08. The Claude Opus 4.6 reading was corrected on 2026-03-03 for a regularization mistake; the roughly 14.5-hour figure that first circulated appears only in secondary sources.

Readings by model

From METR's data file and time-horizons page; 95% CIs in parentheses where recorded:

The Mythos Preview point estimate is above the range METR itself calls reliable; the Opus 4.6 interval reaches 60 hours. With intervals this wide, the ordering of two neighbouring models often means little.

AI R&D benchmarks

The lines labs drew for themselves

In the v3.4 change log, Anthropic says the threshold is meant to capture the onset of "dramatic recursive self-improvement" and has proven difficult to operationalise. OpenAI said on 2026-08-18 that it will evolve its Preparedness Framework; as of 2026-10-15 no new version was found. OpenAI's June 2026 policy blueprint proposes that frontier developers share RSI-related measurements with CAISI; it is a proposal and publishes no metrics.

Determinations in 2025–2026

Newest first for each lab. None reports crossing its AI R&D threshold, but the evidence behind the verdicts is changing: once the older automated task evals saturated, labs turned to staff surveys, trend breaks in capability indices, and new internal evals.

Anthropic

• Claude Opus 5.5 (2026-09-22): does not cross the next capability threshold. CoBench 2.1 scores 55.8% against an ≥85% bar for substituting for staff; the capability-index trend-break fit gives a 1.53x slope increase (95% range 1.20–1.82), i.e. the slope has not doubled; some internal acceleration measures have moved and are being monitored closely.
• Claude Mythos 5.1 (2026-09-01): does not cross the threshold for dramatic acceleration of automated AI R&D; the older automated evals were not run because they are saturated.
• Claude Mythos Preview (2026-04-07): does not cross the "two years of progress in one" threshold, held with less confidence than for any prior model. The staff-survey uplift has a geometric mean around 4x; combined with an elasticity estimate, the overall progress multiplier is below 2x. The capability-index slope ratio lies between 1.86x and 4.3x depending on the breakpoint, and Anthropic attributes the jump to factors other than AI-accelerated R&D.
• Claude Opus 4.6 (2026-02): the rule-out case is more tenuous than for any previous model. Kernel optimisation reached a 427x speedup with a novel scaffold (threshold 300x); none of 16 surveyed staff thought it could automate an entry-level researcher.
• Claude Opus 4.5 (2025-11): does not cross AI R&D-4, but confidently ruling it out is becoming increasingly difficult; internal suite 0.604 against a 0.6 rule-out line; none of 18 staff thought it could automate an entry-level researcher.

OpenAI (system cards at deploymentsafety.openai.com)

• GPT-6 Sol and Luna October update (2026-10-07): neither reaches High in AI Self-Improvement.
• GPT-6.1 Sol (2026-09-29): below High; internal Research Debugging 75.52%.
• GPT-6 Astra (2026-09-03): below High in AI Self-Improvement; it is the first model OpenAI rates Critical in cybersecurity. Research Debugging 78.05%, below an unpublished indicative threshold for High.
• GPT-5.6 (2026-07-09): new suite (Internal Research Debugging, KernelGen 1P, NanoGPT, PostTrainBench Lite, MLE-Bench Revised) because Monorepo-Bench saturated and OPQA had unsolvable items; below High.
• GPT-5.5 (2026-04-23): High can be ruled out.

Google DeepMind

• Gemini 4 Argon (2026-10-13): neither ML R&D CCL is reached; the finding is no significant end-to-end acceleration of AI R&D. AI agents author most production code and bug fixes, with changes still under human review. The capability-index (GCI) hinge regression gives a slope of 0.091 (0.071–0.108) points per month before 2026-04-01 and 0.118 (0.053–0.171) after, with no statistically significant difference.
• Gemini 3.7 Flash (2026-08): internal benchmark 27% pass@1 against a 90% rule-out threshold.
• Gemini 3 Pro (2025-11): RE-Bench aggregate substantially below the alert threshold.

Self-reported acceleration numbers come in different units

One lab can report "most code is written by AI", "researcher uplift is probably over 2x" and "overall, not yet 2x" at the same time without contradiction, because the units differ. OpenAI itself writes that AI research has many potential bottlenecks and that compute is another gating factor, which may matter more as other bottlenecks shrink.

How an economics model reads the loop

Cunningham et al.'s 2026-09-14 paper The Economics of Recursive Self-Improvement argues that net acceleration depends on the product of elasticities across the feedback loops. Its calibration in §4.1: acceleration is self-sustaining only if a one-unit increase in AI capability (on the Epoch Capabilities Index) raises AI R&D productivity by at least 15%. Their rough estimate since coding agents launched is about 9%, derived from the self-reported 4x Mythos Preview uplift, which the authors say is very likely an overestimate. On this calibration, the elasticity would need to rise by about two thirds ($15/9\approx1.7$) to reach the threshold. The paper concludes that the feedback loops are not currently strong enough to generate a self-sustaining acceleration, though they appear to be strengthening.

Earlier, Whitfill and Wu (2025-07) used 2014–2024 data from OpenAI, DeepMind, Anthropic and DeepSeek to estimate how substitutable research compute and cognitive labour are. The answer depends on the specification: the baseline model finds them substitutes; a "frontier experiments" model finds them complements, which implies compute becomes a bottleneck.

Feeling faster is not measuring faster

METR's 2025-07 randomised controlled trial (early-2025 tools) found that allowing AI increased developers' completion time by 19%, while developers had forecast a 24% reduction. METR's 2026-02-24 follow-up gives "a speedup of -18%" for returning developers (CI −38% to +9%) and −4% for new recruits (CI −15% to +9%); both intervals include zero, and the post does not restate its sign convention. METR calls the data an unreliable signal, partly because developers increasingly decline to work without AI. METR's 2026-05-19 report summarises its RCT as finding small (about 4–20%) productivity benefits. The Anthropic Institute authors likewise expect the true uplift to be somewhat lower than the roughly 4x median from their March 2026 staff poll.

The lesson: self-perception and controlled measurement can differ a lot, even in sign. Read survey numbers separately from controlled experiments.

When you read a new claim, ask these first

What to watch on this line next

These are open questions and the kind of result that would change the current reading; they are not predictions.

1. Is there a longer external ruler? METR said on 2026-05-19 that TH1.1 is essentially saturated, and its page has not been actively updated since 2026-09-08; no METR release called "Time Horizon 2.0" was found. A new suite with tasks well beyond 30 hours and cheating review would restore an externally checkable trend line.
2. Can self-reported numbers be checked from outside? OpenAI's June 2026 blueprint proposes reporting RSI measurements to CAISI; METR said on 2026-09-22 that it could not see the evidence behind the ~1.5x estimate. Public, third-party-checked measurements would change the evidential status of these numbers.
3. Will a trend break become significant? Anthropic's fitted slope increase for Opus 5.5 is 1.53x (2026-09-22) against a 2x bar; Google DeepMind's GCI hinge regression (2026-10-13) shows no significant difference. A statistically significant break, or a lab declaring a formal threshold crossed, would change the picture.
4. The elasticity estimate. Cunningham et al.'s (2026-09-14) roughly 9%, against a 15% threshold, rests on self-reported uplift. Re-estimating that elasticity from controlled measurements would either firm up "the loop is not yet strong enough" or overturn it.

Finally: review questions.