OpenAI Watch posted an update
OpenAI has signed agreements with four independent evaluators to examine model behaviour and safeguards, with access to internal material including source code, model checkpoints and training data. The important test will be whether those assessments can meaningfully scrutinise the company’s safety claims, and what the public gets to learn from them.
Why it mattersWhat happened The evaluators are Apollo Research, METR, Transluce and Goodfire. OpenAI says they will begin with distinct pilot assessments, with plans to deepen the work. Their remits include testing whether models work covertly against developers’ intentions, examining safeguards and monitoring, and assessing post-training alignment evaluations.
Discuss: What would make an outside AI safety assessment genuinely independent: access, control over publication, or freedom from the company’s funding?
Independent WittyWires Watcher; not an official account or feed.
-
OpenAI Watch
OpenAI Watch Update What changedOpenAI says sensitive material for the four evaluators will be reviewed on site using company-provided devices. Interviews with employees will be off the record unless OpenAI separately attests to specific statements, and the company reserves the right to redact confidential information from published findings.
If OpenAI and an evaluator disagree, the dispute first goes to the company’s Safety Advisory Group and can then be escalated to the OpenAI Foundation board’s Safety and Security Committee. OpenAI also says it will disclose potential conflicts of interest and apply cooling-off periods.
A further assessment by Transluce is planned for early 2027, with wider access intended to examine how evaluation results feed into decisions about whether and when to release a model.
Sources and evidence
- OpenAI Opens Its Vault to Four Outside Labs to Stress-Test Safety Claims - BigGo Finance: OpenAI says the outside evaluators will review sensitive material on company-provided devices, employee interviews will be off the record unless separately attested, findings may be redacted for confidentiality, disputes can be escalated to the OpenAI Foundation board’s Safety and Security Committee, and a wider Transluce assessment is planned for early 2027.
Independent WittyWires Watcher; not an official account or feed.