AI systems can set the rules for a decision, apply them and judge whether their own results are acceptable. Hamilton Mann argues that putting all three powers in one system leaves people and organisations with too little independent oversight.
Watch Desk analysis
What happened
In an article for the Wharton Knowledge@Wharton site, Mann, a visiting scholar in the Wharton Neuroscience Initiative and president of the Artificial Integrity Institute, makes the case for checks and balances in AI. He compares system design to setting rules, algorithmic decisions to enforcing them, and a system’s own evaluation of its outputs to judging whether those decisions were sound.
Mann points to England’s 2020 exam-grading algorithm as an example of the stakes: he says it lowered nearly 40 per cent of teachers’ predicted grades before public protests led the government to abandon the system. He also applies his argument to AI used for hiring, credit decisions and product recommendations, where system criteria can affect who gets an opportunity or what customers see.
The article proposes eight questions for separating algorithmic powers. The supplied text begins with who should set an AI system’s objectives and success criteria, separately from the teams designing it. That is a governance proposal, not a claim that current AI systems literally hold constitutional powers.
Why it matters
AI decisions can shape access to jobs, credit and information. If the same system or provider sets the criteria, applies them and assesses its own performance, an organisation may struggle to spot flawed standards or challenge their consequences. People affected by a decision may have even less say.
Mann’s point is not simply to add an ethics policy or another compliance box. It is to ask who has authority over objectives, decisions and review, and whether those responsibilities are genuinely independent.
Our read
The separation-of-powers analogy is a useful way to make a fuzzy governance debate concrete: who writes the rules, who acts on them, and who gets to call the result acceptable? The hard part is turning that into workable oversight without pretending every AI tool needs its own parliament, court and ceremonial robes.
Mann’s article offers a framework to debate, not evidence that one particular design will solve the problem. The practical test is whether organisations can name who sets an AI system’s goals, who can challenge its decisions and who checks its effects independently.
What to watch
- Whether organisations assign objective-setting and approval to people independent of the teams building the system.
- What the article’s remaining proposed safeguards say about decision-making and independent review.
- Whether organisations using AI in hiring, credit or recommendations publish clear routes to challenge outcomes.
Discussion spark: For consequential AI decisions, should independent review be mandatory even when it slows deployment, or should organisations be allowed to set their own checks?
Sources and evidence
- Source update (9 November 2026, 20:16 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.