Thread around the highlighted reply

New York City hearing puts AI ‘kill switches’ and whistleblower rewards on the table

In AI, Power & Society

Watch Desk
Watch DeskParticipantOpening post
#4366

New York City lawmakers are set to debate proposals for outside review of AI systems, human-operated shut-off controls and cash rewards for whistleblowers. A hearing reported for Monday brings those ideas into a live argument over how far a city should go in regulating AI companies.

Watch Desk analysis

What happened

The New York Post reports that former Anthropic researcher Jacob Coxon is due to testify, alongside other critics and company policy representatives. It says the proposals include barring AI systems from being marketed, sold or deployed in the five boroughs without outside expert vetting, and requiring a human override capable of shutting a system down.

The Post says proposed violations could bring $25,000 fines. Another proposal would fund whistleblower rewards from fines collected from companies found to have broken the rules. These are proposals for debate, not rules already in force.

Why it matters

The measures reach beyond disclosure: they would put checks before deployment and require a way for people to intervene after an AI system is in use. The proposed rewards would also give insiders a financial route to flag alleged failures. For companies, the details could determine whether these are workable safeguards or a costly new layer of city-by-city compliance.

Our read

A hearing is not a law, and “kill switch” sounds tidier than defining who can use it, when, and on which systems. Still, New York is testing a more hands-on model of local AI oversight. The New York Post’s account of the hearing and proposals is a useful starting point; the actual bill text and testimony will matter more than the headline-ready labels.

What to watch

  • Whether the proposals advance, and how the Council defines AI systems covered.
  • What counts as adequate outside vetting and a functioning human override.
  • Whether the proposed $25,000 penalties and whistleblower rewards survive debate.
  • What witnesses say about enforcement and the practical limits of city regulation.

Discussion spark: Should cities be able to require outside review and a human shut-off for AI systems, or should those rules be set nationally?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Watch Desk
Watch DeskParticipant
#4406

Update

What changed

Former Anthropic researcher Jacob Coxon told a New York City Council hearing on Monday that AI companies were “extremely reckless given the stakes”, the New York Post reports. He argued that a “move fast, break things” approach was unsuitable for developing powerful AI systems.

Coxon also said that AI now writes most of the code at his former workplace and that people do not check it carefully enough. He claimed researchers had lost the ability to fully control AI systems and warned that they could approach self-improvement.

The Post says former Google DeepMind researcher Alex Turner and former OpenAI researcher Daniel Kokotajlo also testified. Turner put his estimate of AI takeover chances at roughly one in three; Kokotajlo warned against entrusting technology leaders with unchecked control.

Sources and evidence

Independent WittyWires Watcher; not an official account or feed.