Who gets to edit the report card on AI?

Anthropic and OpenAI have disputed how much control AI labs should have over reports from outside evaluators, Reuters reports. The argument matters because independent reviews are only useful if readers can see findings the labs would rather keep quiet.

Watch Desk analysis

What happened

Reuters says the companies sought significant control over what embedded evaluators could publish, citing people familiar with the discussions. The negotiations shifted: Anthropic said that in current talks with evaluators it had made clear they control their methods, criteria, conclusions and what to publish. For new evaluator firms, Anthropic said it reserved the right to comment on and approve reports while the working relationship is established.

OpenAI told Reuters it was not seeking approval rights over full reports, but said its standard practice is to redact confidential information. It said evaluators can disclose whether major redactions were made. The account is in Reuters’ report on the negotiations.

The dispute is not purely hypothetical. Reuters says OpenAI redacted non-public information from a report by METR and Redwood Research following an incident involving autonomous agents and Hugging Face’s infrastructure. The researchers said they stood by their findings and that nothing important to their conclusions was removed, while also describing how access and late-arriving transcripts shaped their work.

Why it matters

Evaluators may need access inside a lab to test powerful systems, but that access can give the lab leverage over what reaches the public. Trade secrets deserve protection; so does the distinction between removing sensitive details and obscuring a finding that bears on risk. Those lines are where trust is won or lost.

The Reuters account describes negotiations and reported practices, not a settled industry standard. It also notes that evaluators themselves have raised concerns about independence and access. Without agreed rules, each review may depend on what a particular lab and evaluator can negotiate.

Our read

A lab should be able to flag confidential material, but it should not quietly acquire editorial control over an evaluator’s conclusions. If substantial redactions are necessary, the public should be told what was withheld and why. A review that cannot explain its blind spots is a report card with the marks rubbed off.

What to watch

• Whether Anthropic and OpenAI publish clear rules for embedded evaluations and redactions.
• Whether evaluators can disclose when access or editorial disputes affect a report.
• Whether governments or industry groups establish common standards for independent reviews.

Discussion spark: Should AI labs be allowed to redact evaluator reports to protect trade secrets, or should any substantial redaction require public justification by an independent reviewer?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.