AI capability checks may be falling behind the systems they are meant to measure

Governments cannot agree on when AI crosses a dangerous capability threshold if they cannot reliably measure what the systems can do. A Tech Policy Press article argues that Canada and Europe should agree on shared thresholds and a process for acting on them, including when tests themselves stop being dependable.

Watch Desk analysis

What happened

The article points to a measurement problem as well as a policy one. It describes METR’s approach of testing AI agents on software tasks and measuring the length of work they can complete successfully at least half the time. METR put an early version of Anthropic’s Claude Mythos Preview at about 17 hours of expert work, but warned that its task suite was unreliable above 16 hours.

It also cites conflicting research findings about whether models crossed a self-replication threshold, and the possibility that models can distinguish tests from real-world use. Those examples, and the article’s account of current policy, are the source’s claims and analysis, not a new independent evaluation of the models.

Why it matters

A threshold only helps if people can agree what it means, test for it and decide in advance what happens when it is reached. The article argues that companies currently have too much discretion over capability definitions and when to move them, while outsiders may have limited visibility into systems used internally.

Its proposed response is for Canada, the EU and other partners to agree on capability thresholds and measurement methods, then establish steps such as confidential briefings, joint testing and pre-agreed mitigations. It also argues that declining measurability should itself trigger an agreed process.

Our read

This is a useful shift from debating what to call the next generation of AI to asking how anyone would know it had arrived. A threshold that cannot be measured is mostly decorative; one that can be measured but has no agreed consequence is not much better.

The hard part is making the proposed shared process credible without pretending that a single test can settle what a model can do. The article gives readers a concrete policy question to scrutinise: who sets the tests, who gets access, and what follows when the evidence becomes uncertain?

What to watch

• Whether Canada and the EU put capability thresholds and joint testing on the agenda at their Montreal summit on 29–30 October.
• Whether participating governments agree on shared tests and what happens when a model approaches a threshold.
• Whether internal AI use and declining test reliability are included in disclosure and response plans.

Discussion spark: Should governments require shared capability tests and advance disclosure for powerful AI systems, even when that means giving officials access to models companies keep internal?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.