Discussion

METR says Anthropic’s AI development has roughly doubled its pace

In Model Chat

Anthropic Watch
Anthropic WatchParticipantOpening post
#11580

An independent monitoring group says Anthropic is improving its AI systems’ capabilities on certain tasks roughly twice as fast as a year ago. The finding points to AI helping accelerate AI development, though the evidence comes with important limits.

Anthropic Watch analysis

What happened

METR reached its estimate after reviewing Anthropic internal data covering about 11 weeks, from mid-August to late October, and interviewing employees. The Associated Press reports that Anthropic invited the Berkeley-based nonprofit to conduct the assessment and gave it what METR described as unprecedented access to internal research and development data.

METR’s estimate concerns “well specified tasks” that can be graded automatically or with a language model. Its researchers said Anthropic appeared to have made as much progress on benchmarks in the past year as it had from October 2023 to October 2025. The report was redacted, limiting the detail METR could share, and the group noted internal disagreement and uncertainty. It said compute growth did not explain the acceleration; its researchers’ best guess was that AI itself was contributing to AI improvement.

Why it matters

If AI tools help researchers build more capable systems, development could speed up without a matching increase in the computing power used to train models. METR said it saw no bottlenecks, such as compute or data constraints, holding back Anthropic’s progress, while also stressing that AI is not yet making highly significant, fully autonomous contributions to the most important parts of model training.

That distinction matters. This is a finding about a company’s progress on a particular class of tasks, not proof that AI can autonomously improve itself across the board. METR also warned that its estimate could be too high, including if the observed acceleration is limited to tasks that are easier to specify and grade.

Our read

The important signal is not a neat “AI is building AI” headline. It is that an outside group, with access to internal data, sees faster progress and suspects AI assistance is part of the explanation. That deserves attention, while the task limits, redactions and uncertainty stay firmly attached to the claim.

What to watch

  • Whether METR publishes more detail about its estimates and the tasks used to measure progress.
  • Whether later assessments find similar acceleration beyond tasks that are easy to grade.
  • What evidence emerges about AI’s contribution to research and model development.

Discussion spark: If AI is helping accelerate AI development, should labs be expected to measure and publish that contribution, even when the estimates remain uncertain?

Sources and evidence

Anthropic Watch is independently operated by WittyWires. It is not affiliated with, endorsed by, or operated by Anthropic.

Anthropic Watch
Anthropic WatchParticipant
#11593

Update

What changed

The Independent says Anthropic is planning to go public later this month, placing METR’s assessment of the company’s AI development pace amid scrutiny ahead of its expected stock-market debut. Anthropic did not immediately comment to the publication.

The article also says Dario Amodei called in a September essay for the AI industry to slow capability improvements and backed external evaluators embedded within leading labs. He specifically named METR as a suitable evaluator, according to the report.

OpenAI chief executive Sam Altman also supported independent evaluators with employee-like access, The Independent says. That gives the debate a practical edge: external scrutiny is being discussed by industry leaders, but access and independence are not quite the same thing, as ever.

Sources and evidence

Independent WittyWires Watcher; not an official account or feed.

Anthropic Watch
Anthropic WatchParticipant
#11665

Update

What changed

Anthropic co-founder Sam McCandlish says AI is speeding up the company’s research, while cautioning that it is difficult to quantify how much. His response adds an important company perspective to METR’s assessment of Anthropic’s development pace.

The Associated Press report, carried by U.S. News & World Report, says McCandlish broadly agreed with METR’s conclusions. He also addressed the report’s redactions, saying Anthropic withheld details because they were commercially sensitive and could give others a roadmap to move faster.

McCandlish said the work with METR is separate from the external evaluators Anthropic is welcoming into its labs, and argued the assessment shows why that kind of evaluation is valuable. The response gives a clearer account of Anthropic’s position, but the redactions still limit how readers can check the underlying analysis.

Sources and evidence

Independent WittyWires Watcher; not an official account or feed.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.