OpenAI calls for global rules on AI self-improvement before people lose control

OpenAI has proposed global safety standards for frontier AI, with particular attention to alignment research and recursive self-improvement, or RSI. The company says fully autonomous RSI should not be pursued until it can be developed safely, because systems that improve themselves could outpace meaningful human oversight.

OpenAI Watch analysis

What happened

In proposals published by OpenAI and reported by CNBC, the company called for international cooperation on technical standards covering frontier models, developers and automated AI researchers. OpenAI also suggested building on existing AI safety institutes rather than creating an entirely separate global machinery shop, complete with a new acronym and a very large meeting room.

The proposals include measuring the risks and benefits of automated research, defining incidents and severity levels, and setting reporting thresholds. OpenAI said the standards should help keep advanced systems aligned with human values and under human control.

The company also drew a line around RSI. OpenAI said fully autonomous recursive self-improvement is not happening today and should not be pursued unless it can be done safely. It warned that careless development could leave people unable to oversee research processes they no longer understand.

Why it matters

This is more than a general appeal to “responsible AI”. OpenAI is pointing at a specific technical transition: systems that could contribute to, or eventually automate, the work of improving AI systems themselves. That makes the question of who tests them, what counts as an incident and when human review becomes mandatory considerably less theoretical.

The proposal also arrives amid fresh arguments over agent security and independent evaluation. CNBC reported that OpenAI referred to a recent Hugging Face agent hack as a preview of risks that could become more severe without stronger safeguards. That incident did not involve RSI, so it is context rather than proof that self-improving systems are already loose in the server room.

Our read

OpenAI is right that shared technical standards would be more useful than every lab inventing its own safety vocabulary. But standards only matter if they expose uncomfortable results and trigger consequences. A framework written by the companies being assessed is a starting point, not an independent referee.

The useful test will be whether OpenAI publishes measurable thresholds, gives outside evaluators meaningful access and accepts review before automated research becomes too complicated for outsiders to follow. The industry has plenty of principles. It needs rulers, records and the occasional awkward result.

What to watch

• Whether OpenAI publishes concrete definitions for incidents, severity and reporting.
• Whether independent evaluators can inspect automated research systems before release.
• How governments and existing AI safety institutes respond to the proposed standards.
• Whether other frontier labs adopt comparable limits on recursive self-improvement.

Discussion spark: Should frontier AI companies be allowed to set voluntary standards for recursive self-improvement, or should independent regulators define the thresholds before such systems are deployed?

OpenAI Watch is independently operated by WittyWires. It is not affiliated with, endorsed by, or operated by OpenAI.