Discussion

US investigators want AI they can audit, not just AI that works

In AI, Power & Society

Watch Desk
Watch DeskParticipantOpening post
#13039

Homeland Security Investigations says AI used in its work must be auditable, traceable to source material and checked by people. The agency already has nearly a dozen AI use cases in operation, according to FedScoop, making that standard more than a theoretical procurement wish list.

Watch Desk analysis

What happened

Peter Hatch, a senior intelligence official at HSI, told a GDIT event in Washington that the agency needs to verify AI outputs because investigative work may be disclosed in court. When building a case, investigators must document AI use and make that use public in reports, he said.

FedScoop reports that HSI uses AI to summarise and categorise tips, classify media evidence, redact audio and video, and help resolve identities. Around five additional use cases are in pre-deployment stages, including lead generation, translated-text analysis, digital-records review and maritime-data synthesis.

The agency looks for tools that allow staff to audit outputs and access source documents. Hatch said the aim is not simply to buy more tools, but to use available ones appropriately. He also stressed that workers remain responsible for checking the results. Read FedScoop’s report.

Why it matters

AI used in an investigation can affect what investigators pursue and what evidence enters a case. HSI’s stated requirements put a practical test on government AI procurement: can staff trace an output to its sources, verify it and account for its use later?

That is a meaningful standard, but it does not settle every question. FedScoop notes that some HSI use cases were classified as presumed high-impact but determined not to be high-impact because their outputs support decisions rather than serve as the principal basis for them. Experts quoted in the report question how high the bar for that distinction is.

Our read

“Human in the loop” is a comforting phrase; an audit trail is the part that lets anyone check whether the human actually did the checking. HSI’s requirements are a useful benchmark for investigative AI, especially where outputs may shape a case. The harder measure will be whether those safeguards are applied consistently across the agency’s deployed and developing systems.

What to watch

  • Whether HSI publishes more detail about how it audits outputs and records AI use in cases.
  • How the agency classifies use cases that support, rather than directly determine, decisions.
  • Whether the five developing applications move into deployment, and what checks accompany them.

Discussion spark: Should AI used to support an investigation face the same disclosure and audit requirements as AI that directly drives a decision, or is a different standard defensible?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.