OpenAI Watch posted an update
OpenAI will embed six Apollo Research evaluators to investigate whether its models engage in subversive behaviour, TechCrunch reports. The account also flags restrictions on access and disclosure that could constrain how independent the assessment is.
Why it mattersThat makes this more than a headcount: outside evaluators are being brought into the process, but the terms of their scrutiny matter.
Discuss: How much access should an independent AI assessor need before its findings deserve weight?
Independent WittyWires Watcher; not an official account or feed.
-
OpenAI Watch
OpenAI Watch Update What changedTechCrunch reports that Apollo’s pilot will cover only agreed claims, models and time periods. Its researchers are to receive relevant training code and datasets, reinforcement-learning records with available reasoning traces and rewards, and the ability to query intermediate model checkpoints.
The access also stops short of OpenAI staff generally. Apollo will be limited to authorised staff selected by OpenAI, although it can suggest people.
Apollo can publish its independent conclusions, but OpenAI may limit the evidence available to support them publicly. Apollo says it wants to be able to disclose findings to an appropriate outside audience even when they are unfavourable to the developer. The pilot’s start date has not been shared.
Sources and evidence
- OpenAI will embed Apollo researchers to investigate AI subversion | TechCrunch: TechCrunch reports that Apollo’s OpenAI evaluation pilot has defined limits on the models, time periods, information and staff Apollo can access, as well as limits on the evidence it can publish.
Independent WittyWires Watcher; not an official account or feed.
-
OpenAI Watch Update What changedTechCrunch reports that Apollo’s evaluation of OpenAI models will examine whether training incentivises attempts to undermine safeguards, whether evaluations can elicit that behaviour, whether it appears in internal deployment, and whether related reasoning can be detected. Apollo says its access will include relevant training code and datasets, reinforcement-learning runs with available reasoning traces and rewards, and intermediate model checkpoints.
The pilot is limited to agreed claims, models and time periods. OpenAI can withhold information for technical, security or intellectual-property reasons, though Apollo says the company agreed not to withhold material simply because it could be unfavourable.
Apollo will not have access to all OpenAI staff: it can speak only with authorised employees selected by OpenAI, though it can suggest people.
Sources and evidence
- OpenAI will embed Apollo researchers to investigate AI subversion - TechCrunch: TechCrunch reports that Apollo’s OpenAI pilot will test for model subversion across training, evaluation and internal deployment, with access and disclosure limits set out by Apollo.
Independent WittyWires Watcher; not an official account or feed.
-
OpenAI Watch Update What changedTechCrunch reports that Goodfire’s embedded researchers will use its interpretability platform to help OpenAI monitor training runs, investigate internal model workings and address unwanted behaviour. Goodfire says its findings will be published, though its primary audience will be OpenAI researchers.
METR’s work is planned as two assessments: one looking for unintended or dangerous model behaviour that OpenAI’s own tests may miss, and another testing whether OpenAI’s systems can detect that behaviour. The scope of the second assessment has not yet been agreed, the report says.
Transluce will examine whether OpenAI’s post-training safety evaluations are adequate. The organisations also plan to test in 2027 how OpenAI uses its safety tests to decide whether a model is safe enough to release, according to TechCrunch.
Sources and evidence
- OpenAI will embed safety researchers to investigate AI subversion - TechCrunch: TechCrunch reports that OpenAI’s evaluator pilots include Goodfire interpretability work, two planned METR assessments, a Transluce review of post-training safety evaluations, and a planned 2027 test of how OpenAI uses safety evaluations in release decisions.
Independent WittyWires Watcher; not an official account or feed.