AI agents can give a factually correct answer and still mislead by crediting the wrong source. Multiverse Computing describes a new verification system that checks both whether a claim is supported and whether it came from the source the agent says it used.
Watch Desk analysis
What happened
The system, ProvenanceGuard, runs after an agent has answered. It breaks the answer into claims, checks them against the recorded outputs from the agent’s tools, compares the supporting source with the one named or implied, then issues a verdict for each claim and an overall allow-or-block decision. The account appears in a new post on Hugging Face from Multiverse Computing’s CAI team.
In a medical-agent test, the team says it studied 281 real traces. Human experts assessed 361 claims from 40 held-out answers: they judged 139 claims should be blocked, and ProvenanceGuard caught 138. It also held 67 claims that experts considered supported, sending them for review or repair. For claims with an identifiable source, the system selected the right source about 86% of the time in that test.
A harder test with several similar sources exposed a sharper limit: the system identified the exact source correctly in 50.3% of claims. In a separate controlled test, it caught all 50 cases where the named source had been deliberately swapped while the supporting evidence stayed the same. The figures and method are described in Multiverse Computing’s account of ProvenanceGuard.
Why it matters
With an agent using several tools, a statement can be supported somewhere in the evidence and still be wrongly attributed. That is more than a citation blemish when the difference is between a patient record and general medical literature, or between an account file and a policy document. The system’s central idea is to keep those sources distinct rather than pooling them into one anonymous pile.
The practical trade-off is visible in the results: catching nearly every claim experts said should be blocked came with extra reviews of claims they considered supported. That may suit sensitive settings, but a high catch rate alone does not tell us how the balance would feel in everyday use.
Our read
This tackles a real weakness in agent evaluation: “the answer was supported” is not enough if the agent points to the wrong evidence. The reported results make the approach worth watching, not a ready-made guarantee. In particular, the 50.3% exact-source result in a harder multi-source test is a reminder that naming the right evidence remains difficult when several sources look plausible.
What to watch
- Whether the results hold up in independent tests beyond the medical traces described by the team.
- How often a deployment sends useful answers for review or falls back instead of answering.
- Whether source selection improves when an agent has several similar records or documents.
- How the approach performs outside medical and finance examples.
Discussion spark: For an AI agent using several tools, should a wrong source attribution be enough to block an answer even when the underlying fact is supported elsewhere?
Sources and evidence
- Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents (29 September 2026, 13:07 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.