AI can sift policy evidence, but struggles with the hard choices

Public generative AI tools can help with evidence-gathering for policymaking, but may produce bland proposals that miss political trade-offs, according to a Harvard Kennedy School working paper. The finding is a useful reality check for any government hoping that faster summaries will do the job of policy judgement.

Watch Desk analysis

What happened

The paper, written by Ed Balls, Dylan Evans and Nyasha Weinberg, tested publicly available AI tools on policymaking tasks across three broad areas, then examined regional growth in more depth. The authors also tested how expert checking and the choice of source material affected results. Resultsense describes the findings as indicative, and says the paper is dated 19 October.

In a reading highlighted by Financial Times public policy editor Chris Smyth, the tools were good at summarising evidence and linking datasets, but struggled with difficult questions such as NHS productivity and housing supply. Smyth said their proposals absorbed government spin and offered technocratic variations on existing approaches rather than new ideas or a sound judgement of which policies work.

Why it matters

That split matters for public services weighing AI not just as a writing aid, but as a way to change how policy is made. Faster evidence-gathering could be useful; it is not the same as deciding which trade-offs are acceptable or whether a proposal will work in practice.

The paper does not establish that every AI tool, policymaking task or source set will produce the same results. Its authors call for more direct experimentation with government, alongside changes to workforce, procurement and governance, and stress that public bodies should retain control over internal data.

Our read

The distinction here is sharper than the usual question of whether AI is “good at policy”. It may speed up the legwork while leaving the consequential choices firmly with people. Anyone promising that a chatbot can slim down policymaking should be asked what happens when the answer is politically awkward, not merely well summarised.

What to watch

• Whether the authors publish further results from their policy simulations.
• How government trials measure AI’s contribution to evidence-gathering and policy decisions separately.
• Whether public bodies set clear rules for expert review and the use of internal data.

Discussion spark: If AI can speed up evidence-gathering but struggles with trade-offs, should governments use it in policy design at all, or keep it strictly to research and summarising?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.