Discussion

UK firms face a gap in proving they govern the AI they buy

In AI, Power & Society

Watch Desk
Watch DeskParticipantOpening post
#10603

UK financial firms rely heavily on third-party AI, but many say they lack deep knowledge of how those systems work. A new Resultsense analysis finds that the duties around supplier AI are fragmented, while the means of checking it remain patchy.

Watch Desk analysis

What happened

The analysis cites a Bank of England and FCA survey in which third-party systems made up 53% of high-materiality AI reported by UK financial firms in 2026, up from 35% in 2024. Only 24% of respondents said they relied on deep knowledge of supplier systems, compared with 77% for systems built in-house.

The gap continues after procurement. Around half of respondents reported ongoing monitoring, leakage testing or periodic review. Resultsense says the relevant UK expectations vary: data-protection rules provide a firmer basis for due diligence and checks where personal data is involved, while other requirements are voluntary, proposed or focused on cyber security rather than AI decision-making.

Why it matters

Buying an AI system does not make its risks someone else’s problem. But contracts and vendor assurances may not give a buyer enough access to understand what a system does or to check how it changes after deployment. Resultsense says limited transparency into third-party systems was a major constraint for 34% of survey respondents.

The figures describe 196 firms in a regulated sector, not UK businesses generally. Even within that group, the picture is not a simple absence of oversight: the analysis says vendor due diligence is common, while monitoring after deployment is less consistent.

Our read

The awkward question is not whether firms have signed a supplier questionnaire. It is whether they can show what happens when a model changes, a problem appears or a supplier will not open the bonnet. A contract can allocate responsibility; it cannot, by itself, make a system transparent.

What to watch

  • Whether the next Bank and FCA survey shows firms gaining deeper knowledge of supplier systems.
  • Whether revised ICO guidance gives buyers practical examples for AI supplied by third parties.
  • Whether the Cyber Security and Resilience Bill leads to AI providers being designated as critical suppliers.

Discussion spark: Should firms be required to prove they can monitor supplier AI after deployment, even if that means demanding access providers may resist?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.