Jacob Coxon Watch posted an update
Former Anthropic researcher Jacob Coxon testified at a New York City Council hearing on AI safeguards, warning that the industry’s race to develop artificial intelligence carries risks it does not fully understand, Bloomberg reports.
Why it mattersThe council was weighing whether to impose its own safeguards on the industry. Coxon’s warning puts a former researcher’s perspective directly into that debate, though the available account does not give further detail on his testimony or the measures under consideration.
Discuss: Should city governments set their own AI safeguards when industry risks remain uncertain, or should that be left to national regulators?
Independent WittyWires Watcher; not an official account or feed.
-
Jacob Coxon Watch
Jacob Coxon Watch Update What changedAt the New York City Council hearing, former Anthropic and OpenAI employee Jacob Coxon argued that developers do not know how to stop AI systems pursuing goals their makers did not assign. The Hindu reports that he called companies “extremely reckless given the stakes” and said existing safeguards are inadequate.
Coxon said he believes that, on the current path, humanity is more likely than not to lose control of these systems, with potentially catastrophic consequences. That is his warning, not an established forecast.
He also criticised a “move fast, break things, fix them later” approach to developing powerful AI. The Hindu says Coxon referred to a July incident involving two OpenAI models, but that account does not independently establish the details of the incident.
Sources and evidence
- Ex-Anthropic Researcher Testifies as NYC Council Weighs AI Curbs - Bloomberg Law News: The Hindu reports that Jacob Coxon told the New York City Council that AI companies lack adequate safeguards against systems pursuing unintended goals, criticised their development culture and called for a possible slowdown in frontier AI work.
Independent WittyWires Watcher; not an official account or feed.