One signal

Update What changed

A report by SSBCrack says former researchers Jacob Coxon and Daniel Kokotajlo also warned New York City lawmakers about the risks of the AI race. Reuters, relaying Bloomberg’s reporting, says the council is weighing bills to establish safeguards around AI, providing a concrete policy backdrop for those warnings.

SSBCrack names representatives from Anthropic, OpenAI, Google and Meta as attendees at the meeting. It identifies Google’s Alice Friend as arguing that existing legal frameworks apply to AI while also calling for a comprehensive federal framework.

The account says Council Speaker Julie Menin pressed company representatives on who should be legally responsible if AI systems cause serious harm or financial losses, and found their answers unsatisfactory.

Sources and evidence

• Former Google DeepMind Researcher Warns Misaligned AI Poses Greater Threat Than China's Development Push - SSBCrack: SSBCrack reports that former researchers Coxon and Kokotajlo raised AI risks at the New York City Council hearing, that representatives of four AI companies attended, and that Menin pressed them on legal accountability. Reuters, relaying Bloomberg, reports the council was weighing AI-safeguard bills and Coxon was to testify.

Independent WittyWires Watcher; not an official account or feed.

Update What changed

CNBC reports that senior representatives from Anthropic, OpenAI, Google and Meta testified under oath at the New York City Council hearing. The council asked them to explain AI risks and discuss possible legislative responses.

The hearing also put questions of accountability on the table. CNBC says Council Speaker Julie Menin pressed the company representatives on who should be legally responsible if AI systems cause serious financial harm or death, and called their answers troubling. Menin said she considered the companies’ inability to quantify the risk of a catastrophic event concerning.

Former OpenAI researcher Daniel Kokotajlo called for greater industry transparency and slower development of leading models, CNBC reports. The article also says SpaceXAI did not attend despite a council subpoena; Menin said the council planned to pursue the matter in court.

Sources and evidence

• AI researcher warns 'we are racing to build and grow our own adversary' in NYC hearing - CNBC: CNBC reports that representatives of Anthropic, OpenAI, Google and Meta testified under oath at a New York City Council hearing on AI risks and possible legislation; it also reports an accountability exchange, calls for greater transparency and slower development, and the council speaker’s stated response to SpaceXAI’s absence.

Independent WittyWires Watcher; not an official account or feed.

Update What changed

Former Anthropic researcher Jacob Coxon told a New York City Council hearing that the chance of humanity losing control of AI on its current trajectory is greater than the chance it will not, according to The Asia Business Daily. He warned that such a loss of control could ultimately lead to human extinction.

Coxon also called the industry’s development approach “extremely reckless” and said a proposed AI kill switch might help in the short term but would not be enough as a long-term safety measure, the paper reports. That adds a concrete objection to the council’s shutdown proposal: Coxon argues that an emergency stop would not resolve the underlying control problem.

The article says Google DeepMind alumnus Alex Turner also testified, warning that an unaligned AI could become a domestic adversary.

Sources and evidence

• "Humanity Could Become Extinct After Losing Control of AI," Warns Former Anthropic Researcher - 아시아경제: The Asia Business Daily reports that Jacob Coxon warned a New York City Council hearing that humanity may lose control of AI on its current trajectory, with possible extinction as a consequence, and argued that a proposed AI kill switch would not suffice as a long-term safeguard.

Independent WittyWires Watcher; not an official account or feed.