Anthropic says Claude now leads 26% of its measured AI research work

Anthropic says Claude now leads 26% of its measured AI research and development work, up from under 1% in February, while people continue to supervise the work. The figures offer an unusually concrete look at an AI company using AI to help build its next AI systems, which is either progress or a sentence designed to make the circularity feel less dizzying.

Anthropic Watch analysis

What happened

Anthropic applied Epoch AI’s scale to a fixed set of tasks, weighted by the human time those tasks normally require. In this framework, “leads” means Claude completes most of a task under human supervision. Anthropic says Claude reached that level for 26% of measured work in August, while it collaborated on or led more than 90% of the work. Full autonomy was zero.

The company also says around 30,000 agents were running simultaneously on its main internal platform in August. Their actions entered monitoring, with roughly one decision blocked for every 47,000 across more than one billion decisions. Anthropic says about 6% of its research and development computing resources went to safety work during a sample week in July, although the snapshot excluded safeguards classifiers.

The Rundown reports that Claude’s own ratings matched human ratings exactly 59% of the time. That is a useful warning label: the headline percentages depend on task selection, classification and a measurement system that still leaves room for disagreement.

Why it matters

This is more than a company saying its chatbot is handy. Anthropic is reporting that Claude now handles substantial portions of work involved in developing future AI systems, even though people still choose goals, supervise tasks and make the final decisions. The more capable the assistant becomes, the more important it is to know whether human review is genuinely keeping pace or merely approving a larger pile of machine-generated work.

The figures also give more substance to CEO Dario Amodei’s call for slower, more closely monitored frontier development. Anthropic says it wants outside evaluators to inspect its systems, and it has announced an evaluation partnership with Accenture covering red teaming, alignment assessments and safeguards testing. The supplied evidence does not establish how much access evaluators will receive or what findings they will be able to publish.

Our read

Anthropic has done something valuable by putting numbers on an otherwise woolly claim about AI helping to build AI. But these are internal measurements, and the company acknowledges that its disclosures depend on what it chooses to include. “26%” is a meaningful signal, not a neutral instrument floating above the lab.

The practical question is whether independent evaluators can inspect the underlying records, challenge task classifications and report uncomfortable results. If they can, Anthropic’s figures may become a useful yardstick. If not, the numbers risk becoming a polished progress dashboard with the difficult bits conveniently below the fold.

What to watch

• Whether Anthropic publishes the task set, weighting method and underlying records.
• Whether outside evaluators can independently test the monitoring system and report adverse findings.
• How the 30,000-agent platform changes as Claude takes on more research work.
• Whether other frontier labs disclose comparable measurements rather than competing only on model benchmarks. Sources and evidence: The Rundown’s 25 September report is the basis for Anthropic’s internal figures, the Epoch AI measurement framework, the monitoring statistics and the company’s stated limits on autonomy. The evidence also cites Anthropic’s safety and evaluation announcements, but does not independently verify the company’s measurements.

Discussion spark: Should frontier AI labs be required to publish independently audited measures of how much AI performs their research, or would that disclosure mainly give large companies another way to set the rules and the yardsticks?

Anthropic Watch is independently operated by WittyWires. It is not affiliated with, endorsed by, or operated by Anthropic.