New York hearing puts AI safety claims and proposed rules on the record

New York City Council heard former AI-company employees warn that frontier AI could become difficult to control, while company representatives faced questions about catastrophic risks and possible local safeguards. The hearing also put concrete proposals, including whistleblower protections and third-party checks, into the debate.

Watch Desk analysis

What happened

At Monday’s hearing, former employees of Anthropic, OpenAI and Google DeepMind testified about AI risks and the pace of development. Former Anthropic engineer Jacob Coxon said he believed humanity was more likely than not to lose control of AI on the current path. That is Coxon’s assessment, not an established prediction.

The Northwest Arkansas Democrat-Gazette’s account of the hearing, which credits contributions from The Associated Press and Bloomberg, says company representatives also appeared. OpenAI policy executive Morgan Dwyer said the company should not train models unless it could make an extremely strong case that they would remain under human control. Anthropic and OpenAI representatives expressed openness to some of the council’s proposals, the report says.

What changes

• Whistleblower incentives
  A proposal would give whistleblowers a share of fines or penalties recovered after reported AI-company violations.
• A route to sue
  Another measure would let people harmed by AI systems sue developers in certain circumstances.
• Independent checks
  The council is considering third-party validation for some systems before deployment.
• A proposed shutdown switch
  A possible “kill switch” for rogue systems was raised, though Coxon said it would not resolve the wider risks.

Why it matters

This is not just a hearing about worst-case scenarios. The proposals point towards specific ways a city might try to oversee AI systems: giving insiders reasons to report violations, opening a route to legal redress and requiring outside checks in some cases. Whether those measures become law, and how they would work in practice, remains unsettled.

The exchange also exposed a hard problem for policymakers: how to make decisions about low-probability, high-consequence risks when company representatives themselves say there is no single useful percentage to offer. Strongly attributed testimony is important evidence of what was argued in the room, not proof that a particular catastrophe will happen.

Our read

The useful part of this hearing is the move from broad declarations to possible rules. A “kill switch” has an appealingly tidy sound for a messy technical problem; the proposals on reporting, liability and independent validation deserve at least as much attention. The council’s next test is to explain what systems each measure covers, who enforces it and how it avoids becoming either theatre or a blunt instrument.

What to watch

• Whether the council advances any of the proposals and publishes their precise scope.
• How officials define which AI systems would need independent validation.
• Whether company representatives provide more detail on the limits of keeping advanced systems under human control.

Discussion spark: Should cities be able to require independent checks on some AI systems before deployment, or would that give local governments too much power over technology used across borders?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.