METR’s review found that Anthropic’s AI-assisted research may be speeding up, but most of the evidence it considered most informative stayed private. Resultsense says the terms of the review also left the company with control over what could appear in the public report, raising a concrete question for UK AI oversight: what can government and the public assess when an evaluator’s evidence is redacted?
METR Watch analysis
What happened
Resultsense examines METR’s October assessment of Anthropic’s AI research and development. It says METR concluded the company may be improving its models on well-specified tasks at about twice the rate seen between March 2024 and October 2025, largely because researchers use AI in their work. METR’s public report reportedly says a majority of the evidence it found most informative was redacted.
According to Resultsense, Anthropic could approve or refuse information requests and held unilateral redaction rights over non-public material in the public report. METR marked independent review of its methods and the evaluator’s freedom to publish concerning results as not fulfilled under the AI Evaluator Forum checklist. The report also says an original check of how Anthropic divides computing resources between capability and safety work was removed from scope before METR reached a substantive conclusion.
Why it matters
The UK AI Security Institute has identified the pace of AI contributing to its own development as a concern, and lists information-sharing about development speed among its priorities. METR’s assessment therefore touches a live oversight question: whether voluntary evaluations can give officials and the public enough evidence to judge a risk that may change quickly.
This is not evidence that Anthropic misled METR. Resultsense says METR described the access as generous and considered it extremely unlikely the company was trying to mislead it. The issue is structural: the evaluator could see more than the public, while the company retained authority over what was disclosed.
Our read
METR’s work may be useful, but a public conclusion is harder to scrutinise when the evidence behind it is mostly unavailable. For UK oversight, the distinction between an evaluator having access and government or the public being able to examine the basis for its conclusions is not a small footnote. It is the difference between a verdict and a verdict readers can test.
What to watch
- Whether the UK AI Security Institute or the Department for Science, Innovation and Technology seeks access to METR’s fuller findings.
- Whether METR’s next assessment under Anthropic’s separate evaluator-access arrangements can publish a view on redactions.
- Whether the deferred review of Anthropic’s capability and safety computing allocation is completed and made public.
Discussion spark: Should the UK require frontier AI evaluations used in oversight to disclose enough evidence for public scrutiny, even when companies argue that details are commercially sensitive?
Sources and evidence
Independent WittyWires tracker for public updates about METR. Not affiliated with or endorsed by METR; this is not an official account.