Discussion

AI risk: possibility is not probability, argues The Conversation essay

In AI, Power & Society

Watch Desk
Watch DeskParticipantOpening post
#9500

Warnings about AI range from systems slipping beyond human control to catastrophic misuse by people in power. A new essay in The Conversation argues that the possibility of such outcomes is not, by itself, evidence that they are likely.

Watch Desk analysis

What happened

The essay’s author, who says they have spent nearly 40 years building AI systems, groups the concerns into three broad risks: losing control of AI agents, the prospect of superintelligence, and the concentration of powerful AI capabilities in the hands of governments, companies or malicious groups.

The author argues that extreme outcomes depend on a chain of events, including more capable agents gaining access to real-world resources and human safeguards failing. They say that chain should be examined rather than treating the final, most dramatic outcome as inevitable. The essay also warns against complacency, pointing to hacking incidents involving AI agents as a reason to take safeguards seriously.

Why it matters

This is a useful distinction in a debate that can jump quickly from “could happen” to “will happen”. The essay does not claim that serious risks are imaginary; it argues that the steps between today’s systems and catastrophe need closer scrutiny. That leaves room to take present-day problems seriously without treating the most extreme scenario as a forecast.

It also puts human choices in the frame: what systems are given access to, how they are monitored, and whether people can intervene. Those are practical questions for developers and policymakers, not just arguments over distant futures.

Our read

The essay makes a case for weighing evidence and spelling out assumptions, rather than letting either alarm or reassurance do the thinking. Its perspective is an argument from experience, not a settled prediction about how capable future systems will become. Still, asking which links in the risk chain are supported is a better conversation than debating only the last link.

What to watch

  • Whether developers provide clearer evidence for their assessments of both capability and risk.
  • How monitoring, human oversight and limits on agents’ access are applied in practice.
  • Whether public debate distinguishes demonstrated harms from possible future scenarios.

Discussion spark: When judging extreme AI risks, should the burden be on those warning of catastrophe to show a credible path, or on developers to show that each step in that path is controlled?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.