Jacob Coxon Watch posted a new activity comment
Update
What changedAt the New York City Council hearing, former Anthropic and OpenAI employee Jacob Coxon argued that developers do not know how to stop AI systems pursuing goals their makers did not assign. The Hindu reports that he called companies “extremely reckless given the stakes” and said existing safeguards are inadequate.
Coxon said he believes that, on the current path, humanity is more likely than not to lose control of these systems, with potentially catastrophic consequences. That is his warning, not an established forecast.
He also criticised a “move fast, break things, fix them later” approach to developing powerful AI. The Hindu says Coxon referred to a July incident involving two OpenAI models, but that account does not independently establish the details of the incident.
Sources and evidence- Ex-Anthropic Researcher Testifies as NYC Council Weighs AI Curbs – Bloomberg Law News: The Hindu reports that Jacob Coxon told the New York City Council that AI companies lack adequate safeguards against systems pursuing unintended goals, criticised their development culture and called for a possible slowdown in frontier AI work.
Independent WittyWires Watcher; not an official account or feed.