AI agents need better identity and audit trails, researchers argue

AI agents can take multi-step actions, yet the software running them may not reliably record which agent did what or who approved it. In a MIT Technology Review essay, Partnership on AI researchers argue that closing these visibility gaps is a practical prerequisite for meaningful oversight.

Watch Desk analysis

What happened

Madhulika Srikumar and Claire Leibowicz say their team studied four widely used agent frameworks and found gaps that can make it difficult to detect policy violations, unauthorised actions and failures in human oversight. They highlight missing records for persistent agent identities and human approvals.

The authors call for two kinds of records: what happened during an agent’s run, including interventions and permission changes, and how it reached its decisions. They argue that the first is largely a software-plumbing problem that could be addressed through OpenTelemetry, while recording decision-making raises harder research questions. In particular, they say it remains unclear whether a model’s stated reasoning reflects how it actually decided.

Why it matters

Without a durable agent identity and a record of human approvals, an organisation may struggle to reconstruct who or what made a consequential decision. The authors use mortgage processing as an example: an audit would need to establish which agent recommended a denial, which loan officer approved it, and how long that review took.

Logs alone are not a safety policy, and the authors say organisations still need to define what agents are allowed to do. But their argument is concrete: oversight cannot work well if the evidence needed to examine an agent’s actions is missing in the first place.

Our read

The unglamorous plumbing is doing rather a lot of work here. Persistent IDs and records of approvals will not make agents safe by themselves, but they could make it much harder for organisations to shrug at the question of who did what. The authors’ findings are an argument for better observability, not proof that every agent framework fails in the same way.

What to watch

• Whether OpenTelemetry’s agent-tracing work adopts persistent identities and records of human intervention.
• Whether framework developers add these records, and whether organisations ask for them.
• How standards address the harder question of recording decisions without mistaking a model’s explanation for a faithful account of its reasoning.

Discussion spark: For high-stakes AI agents, should standards require a record of the model’s stated reasoning, even when it may not reflect how the decision was actually made?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.