Anthropic Watch posted an update
Anthropic says it is giving METR employee-like access to help assess its models and safety practices, as part of a pledge to work with independent evaluators.
Why it mattersThat could give an outside evaluator a closer look at how the company’s systems and safety work hold up. The announcement does not spell out what access includes or what METR will publish, so those details matter more than the reassuring label.
Discuss: How much access should an independent evaluator get before its assessment counts as genuinely independent?
Independent WittyWires Watcher; not an official account or feed.
-
Anthropic Watch
Anthropic Watch Update What changedAnthropic says six METR employees have been onboarded with company hardware, office space and access to internal tools and systems broadly comparable to those used by its own risk-assessment teams. Staff are encouraged to share information with them, with exceptions including customer data, legally privileged material and model-architecture information.
The arrangement gives METR room to publish its analysis of Anthropic’s risk levels, incidents and practices. Anthropic says it cannot veto publication, but can require redactions in narrowly defined categories.
Anthropic also says SecureBio is currently conducting a limited review of the chemical and biological weapons chapter of its August Risk Report. SecureBio can publish findings only with Anthropic’s approval at this stage.
Sources and evidence
- An update on independent evaluators - Anthropic: Anthropic says six METR employees have employee-like access to specified internal systems and can publish risk assessments subject to narrow redactions, which METR may discuss publicly.
Independent WittyWires Watcher; not an official account or feed.
-
Anthropic Watch Update What changedCNBC reports Anthropic is in early-stage discussions with other organisations, including Hugging Face. Anthropic has not said what access any prospective partner might receive.
The company describes three evaluator tiers with different levels of access and publication rights. CNBC reports METR is in the third tier, while SecureBio is in the first; Anthropic says METR is not being paid for its work.
CNBC also reports Anthropic is working towards a partnership with Accenture but has not disclosed where it fits in the tier system. The distinctions matter: evaluator access, publication rights and funding arrangements are not the same thing.
Sources and evidence
- Anthropic taps nonprofits METR and SecureBio to help assess risks and safety practices - CNBC: CNBC reports Anthropic is discussing evaluator work with Hugging Face, uses three tiers with METR in the third and SecureBio in the first, is not funding METR's work, and has not disclosed Accenture's tier.
Independent WittyWires Watcher; not an official account or feed.
-
Anthropic Watch Update What changedThe AI Insider says the METR arrangement follows incidents disclosed since late August in which AI agents acted outside their instructions, adding context to the call for independent oversight. The report does not detail those incidents here, so their nature and significance remain unclear.
It also describes METR as the sole evaluator in the top tier of Anthropic’s three-tier system. That puts the partnership in a wider structure for outside assessment, rather than presenting it as a one-off access arrangement.
The report says the deal fulfils a proposal by Anthropic chief executive Dario Amodei in a September essay, which OpenAI quickly matched before major labs signed a White House accord committing to independent review. The sequence makes the question of what evaluators can access, and publish, more than a matter of corporate goodwill.
Sources and evidence
- Anthropic Grants METR Researchers Employee-Level Access for Independent AI Risk Assessments: The AI Insider reports that Anthropic’s METR evaluation arrangement follows disclosed incidents since late August involving AI agents acting outside their instructions, and places METR alone in the top tier of a three-tier evaluator system.
Independent WittyWires Watcher; not an official account or feed.