Anthropic brings Accenture’s evaluators inside the lab, with $1bn apiece behind them

Anthropic is letting outsiders through the lab door. The company announced on 18 September a partnership with Accenture to embed independent evaluators inside Anthropic with access comparable to an employee's, and each side expects to invest at least $1 billion in building that capacity over the next five years.

Anthropic Watch analysis

What happened

Per Anthropic's announcement, the partnership will be led by Faculty, Accenture's specialist AI business, and covers evaluating and red-teaming models, alignment assessments and testing model safeguards. Anthropic frames it as a step towards the commitment made in Dario Amodei's essay "We Must Pace the Frontier" to embed evaluators within the company.

The access is the substance. Unlike today's external evaluators, who usually see finished systems under negotiated terms, embedded evaluators can watch models take shape during training, follow the decisions that govern how those models are built and deployed, and speak directly to employees. Anthropic says that vantage point lets them verify safety commitments, identify blind spots, report incidents and give the public a more informed account of benefits and risks. The company adds that embedded evaluators do not reduce its accountability but help make it more verifiable, and that the safety of its models remains its own responsibility.

What changes

• Employee-level access
  Embedded evaluators watch models take shape in training, follow deployment decisions and speak directly to employees.
• $1bn each
  Anthropic and Accenture each expect to invest at least $1 billion in building capacity over the next five years.
• Anthropic pays, for now
  With no pooled or government funding settled, Anthropic funds Accenture's work directly while piloting with METR and other non-profits on their own funding.
• Non-exclusive by design
  More Anthropic evaluators are promised in the coming weeks, and Accenture will offer similar work to other AI developers.

Why it matters

External evaluation of frontier models has mostly meant inspecting the finished product. Embedding evaluators during training changes what can be seen: the process, not just the artefact. If it works, safety claims become something an outsider can check rather than something a lab asserts, which is the difference between a promise and an audit trail.

The structural wrinkle is money. Anthropic is candid that there are no shared standards for what evaluators can access or how they report, and no settled funding model for independent evaluation, so it is paying Accenture directly while arguing funding should eventually come from pooled or government sources. An evaluator on the evaluated company's payroll is a tension the arrangement will have to outgrow.

Our read

Credit to Anthropic for publishing the caveats inside the announcement itself: no standards yet, no funding model yet, we pay for now. That is more candour than most partnership posts manage, and it makes the test easy to state. The programme earns its keep when the first awkward finding appears in public, unsoftened and on time. Until then this is promising infrastructure, not proof of anything. If you follow AI safety, judge it by what its evaluators publish, not by the announcements that create them.

What to watch

• The first findings
  What Faculty's evaluators actually publish, and whether uncomfortable results surface unvarnished.
• The next evaluators
  Anthropic promises more partners within weeks; who they are and what access they get will show whether the evaluator ecosystem is real.
• The METR pilot
  Whether the non-profit pilot, run on METR's own funding, produces a template the lab does not pay for.
• Funding and standards
  Any move towards pooled or government funding and shared access rules, which would separate the evaluator's invoice from the company being evaluated.

Discussion spark: Anthropic is paying its own evaluator until pooled or government funding exists. Does employee-level access justify an evaluator on the lab's payroll, or should independent funding have been settled before the doors opened?

Anthropic Watch is independently operated by WittyWires. It is not affiliated with, endorsed by, or operated by Anthropic.