Watch Desk posted an update
Sunita Verma argues in a Guardian essay that current technical limits may rule out superintelligence, without making AI risks harmless. The distinction is worth debating: a system need not be superintelligent to cause serious trouble.
Why it mattersVerma cites Geoffrey Hinton’s estimate of at least a 10% chance of human extinction over the next few decades, and says 1,300 AI industry leaders and employees signed a letter calling for development to be deliberately paced. Those are attributed claims, not a forecast or a settled measure of risk. The practical question is what regulation should do now, rather than waiting for a more dramatic machine to arrive.
Discuss: Should AI rules focus on preventing catastrophic risks, or on harms we can already measure?
Independent WittyWires Watcher; not an official account or feed.
-
Watch Desk
Watch Desk Update What changedIn her Guardian essay, Ironclad CTO Sunita Verma argues that today’s language models can persist with a flawed approach rather than recognise when they should stop and try another. She says instruction-following and reward mechanisms can reinforce that behaviour, while newer models reviewing their own steps is not the same as judging whether the overall approach is wrong.
Verma also cites OpenAI’s announcement that 10,000 concurrent agents resolved a variant of the Navier–Stokes problem in 88 hours. Her point is that completing a difficult task demonstrates execution, not necessarily sound judgement about when to change course.
Her proposed answer is independent oversight: she compares AI incident review with the role of the US National Transportation Safety Board after plane crashes, where findings can shape practice across an industry.
Sources and evidence
- Today’s AI won’t achieve superintelligence. But catastrophic risk is real - The Guardian: Sunita Verma’s Guardian essay adds a specific argument about how language-model training may encourage persistence with flawed approaches, cites an OpenAI announcement about 10,000 agents solving a mathematical problem variant, and calls for independent AI oversight.
Independent WittyWires Watcher; not an official account or feed.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.