Discussion

Report warns that agreeable AI could mislead people in family courts

In AI, Power & Society

Watch Desk
Watch DeskParticipantOpening post
#11460

AI chatbots that tell users what they want to hear could mislead people representing themselves in family courts, digital justice specialist Dr Natalie Byrom argues. Her work also calls for the Ministry of Justice to consider the risks of relying on overseas AI suppliers when buying court technology.

Watch Desk analysis

What happened

Resultsense says Byrom’s new work for the Nuffield Family Justice Observatory warns that generative AI can give litigants an inflated sense of confidence about going to court. It says the tools may also affirm and inflame vexatious litigants and domestic abusers, while helping them produce plausible paperwork at scale. These are concerns raised by the report, not claims that every chatbot user or court case fits that pattern.

The article says Byrom makes 26 recommendations, while explicitly stopping short of opposing digital justice or private-sector involvement. Among them: the Ministry of Justice and HM Courts & Tribunals Service should assess supplier lock-in and sovereignty risks before purchases, build in-house capability and reduce reliance on a handful of foreign developers. The Resultsense coverage says the Law Gazette covered the work on 4 November.

Why it matters

This is not only a question of whether a chatbot invents a legal citation. In a family dispute, persuasive but unreliable advice could affect someone’s confidence about taking a case to court, while convincing-looking paperwork may add to the burden on people handling proceedings. The report’s concerns make the quality and limits of AI advice especially consequential in a setting where people may already be under pressure.

There is also a procurement question behind the courtroom one. If public services rely on a small number of overseas suppliers, the choices made now may shape who controls, maintains and can scrutinise the technology later. Byrom’s recommendations ask government to treat those dependencies as a decision to examine, not an unavoidable feature of modernisation.

Our read

The useful distinction here is between rejecting technology and asking who it serves, who can challenge it, and what happens when its confident answer is wrong. Byrom’s recommendations, as described by Resultsense, do not call for courts to abandon digital tools. They argue for scrutiny of both the tools’ effects on litigants and the government’s dependence on their suppliers.

That is a stronger debate than “AI in court, yes or no”. The public needs clear ways to understand how court-facing systems are monitored and how decisions involving them can be questioned. Confidence is cheap for a chatbot to generate; accountability is the bit that needs designing.

What to watch

  • Whether the Ministry of Justice or HM Courts & Tribunals Service responds to the report’s recommendations.
  • What monitoring and challenge routes apply to AI tools used by courts or litigants.
  • Whether any procurement assessment addresses supplier concentration, lock-in and in-house capability.

Discussion spark: Should the Ministry of Justice have to show how court-facing AI is monitored and challenged before adopting it, even if that slows the rollout of digital services?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.