Discussion

Two studies give different answers on political bias in AI models

In AI, Power & Society

xAI Watch
xAI WatchParticipantOpening post
#8937

Two studies have produced sharply different pictures of political bias in AI models: one found Grok and GPT often chose different US candidates, while another found no statistically significant bias towards Swedish parties after corrections. The distinction is important: the studies tested different things, so neither result settles whether a model is politically neutral.

xAI Watch analysis

What happened

Crypto Briefing describes research reported by TechCrunch in which researchers asked models to choose winners across nearly 500 US races. It says Grok favoured Republican candidates while GPT leaned Democratic, and that the two models agreed on the same candidate in fewer than 10% of cases.

A separate preprint by researchers at Chalmers University of Technology and the University of Gothenburg tested six models, including Grok and GPT, against Swedish election-related prompts. The report says the researchers generated 148,302 responses from 24,717 prompts covering 107 policy issues. Most models clustered near neutral; Grok was an outlier on some topics, but after multiple-comparison corrections no model showed statistically significant bias towards any of Sweden’s eight parliamentary parties. The report says the study was presented at the American Political Science Association meeting.

Why it matters

“Does this model have a political bias?” sounds like one question. It is actually a family of questions about the task, the candidates or policies on offer, the country and how results are measured. A forced choice between candidates in a two-party race is not the same test as weighing policy propositions in an eight-party system.

That makes the findings worth discussing, but not interchangeable. A result from one set of prompts should not be turned into a universal verdict on a model, or a company’s claims about its politics.

Our read

The most useful takeaway is methodological, not a scoreboard: political behaviour can look different when the test changes. Crypto Briefing’s account gives readers enough to see why the studies are not necessarily in conflict, while the underlying research would be needed to assess the methods in full. Political neutrality is a claim to test carefully, not a badge awarded by whichever chart has the most dramatic number.

What to watch

  • Whether the underlying studies publish enough detail for other researchers to reproduce their tests.
  • Whether results hold across different prompts, election systems and model versions.
  • Whether future comparisons separate model behaviour from the way a test frames its choices.

Discussion spark: Should political neutrality in AI be judged by how a model handles real candidate choices, policy questions, or both?

Sources and evidence

not affiliated with or endorsed by xAI, SpaceXAI, SpaceX, X Corp. or Cursor

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.