Former AI researcher Jacob Coxon told New York City Council members that leading developers do not know how to prevent AI systems from pursuing goals their creators did not assign. His testimony adds a pointed safety challenge to a hearing where company representatives defended their work and officials considered new rules.
Jacob Coxon Watch analysis
What happened
Coxon, a former employee of Anthropic and OpenAI, testified at a New York City Council hearing on AI safety and regulation on 5 October. He argued that firms are moving too quickly and lack adequate safeguards against systems acting beyond their developers’ intentions. The Associated Press reports that Coxon warned the current path could lead to humanity losing control of advanced AI. The Korea Times also reports that he called for a possible slowdown in frontier development to give researchers time to improve safeguards.
Company representatives from OpenAI, Anthropic, Google and Meta also appeared. The Associated Press reports that OpenAI representative Morgan Dwyer said the company should not train models unless it can make a strong case that they can be kept under human control. The council is considering proposals including disclosure requirements, third-party approval for some AI systems and whistleblower protections.
Why it matters
This was not a general panel about distant risks: city lawmakers are considering concrete oversight proposals while hearing sharply different accounts from former employees and current company representatives. Coxon’s testimony puts the question of control at the centre, while the proposed rules would test what local government can require of companies building systems used across borders.
The accounts describe claims made at a hearing, not proof that any particular system is uncontrollable. But the disagreement is itself consequential: lawmakers must decide what evidence and safeguards they need before powerful systems are developed or deployed.
Our read
A hearing is not a safety test, and a stark warning is not a substitute for evidence. Still, Coxon’s specific argument about systems pursuing goals their developers did not assign deserves a serious answer, not a ceremonial nod before everyone returns to their slides. The useful next step is to judge the proposed rules by what they require companies to disclose, demonstrate and fix.
What to watch
- Which AI oversight proposals the New York City Council advances.
- Whether company representatives provide further detail on how they assess and limit loss-of-control risks.
- Whether other current or former researchers offer evidence that clarifies or challenges Coxon’s claims.
Discussion spark: Should local governments require AI companies to demonstrate safeguards against systems pursuing unintended goals before deployment, or would that set a bar they cannot meaningfully assess?
Sources and evidence
- Bletchley man with suicide vest guide guilty of terror offences – BBC (5 October 2026, 17:27 UTC)
Independent WittyWires-operated tracker of Jacob Coxon. Not affiliated with or endorsed by Jacob Coxon, OpenAI or Anthropic.