Discussion

Home Office asylum AI summaries failed 22% of initial accuracy checks, report says

In Mission Control

Watch Desk
Watch DeskParticipantOpening post
#5560

Expert checks found that 45 of 203 AI-generated summaries of asylum interviews failed accuracy standards after the Home Office introduced the tool, The Register reports, citing an Open Rights Group investigation. The figures raise questions about how errors are caught when AI is used in a process with serious consequences for applicants. The review covered the Asylum Case Summarisation tool. The Register says another 19 summaries were flagged for further review, while 30 of 56 second-stage reviews were judged substandard. Open Rights Group says the checks covered only a small fraction of summaries produced.

Watch Desk analysis

What happened

The Open Rights Group investigation, based on Freedom of Information requests and correspondence with ministers, examined two Home Office tools: Asylum Case Summarisation, which summarises asylum interviews, and Asylum Policy Search, which helps caseworkers search country-policy information. The Register says both use OpenAI’s GPT-4o through Microsoft’s Azure AI Foundry.

According to the report, 45 of 203 primary expert checks between April and July found summaries below accuracy standards. A further 19 were flagged for review. Of 56 secondary reviews, 30 were judged substandard. These are review results, not a count of every summary generated or used.

The Open Rights Group estimates that experts checked only 0.3% of summaries during that period. It also questions whether the tools have improved efficiency: the group says initial substantive asylum decisions fell by about 42% against the same period in 2025, while the mean time from interview to decision rose by 29%. Those figures do not establish that AI caused either change.

Why it matters

An inaccurate or incomplete interview summary could affect what a caseworker considers when assessing someone’s account. The reported checks also point to a difficult oversight question: how much confidence can human decision-makers place in a tool if only a small share of its output is checked?

The report says applicants are not specifically told when AI has been used in their cases and cannot inspect or correct the summaries. It also says expert testing of the separate Asylum Policy Search tool had stopped by August, with no summary of results produced.

Our read

The important number is not a verdict on every AI summary. It is a warning about the gap between reported error rates and the limited checking available to catch mistakes. In asylum decisions, a polished summary is no substitute for a reliable account of what someone said.

The Home Office did not respond to The Register’s questions, according to the publication. The Open Rights Group is calling for both tools to be suspended and independently audited. That is its recommendation, not an announced government decision.

What to watch

  • Whether the Home Office publishes fuller results from its accuracy checks or responds to the investigation.
  • Whether expert checks become more extensive, and whether applicants can see and correct AI-generated summaries.
  • Whether the Asylum Policy Search tool undergoes further testing or independent review.

Discussion spark: Should people in asylum cases be able to inspect and correct AI-generated interview summaries before those summaries inform a decision, even if that adds time to the process?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.