OpenAI Scientist Urges Global Safety Bars as AI Nears Self-Improvement

OpenAI chief scientist Jakub Pachocki has issued a stark warning that the current pace of artificial intelligence development demands far greater caution than the industry is exercising. In a statement, he said society is not ready for the consequences of machine intelligence that is rapidly surpassing human abilities in transformative fields.

Pachocki pointed to internal research showing that reasoning models are already pushing scientific boundaries, operating digital interfaces, and reshaping computer security. He noted that progress could soon enable recursive self-improvement, where systems drive their own development through substantial capability jumps.

"This is a time that calls for extreme caution," Pachocki said. "I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence."

He acknowledged that OpenAI will continue seeking technical fixes for alignment and monitoring, and may unilaterally pause scaling if needed. However, he argued that such measures alone are insufficient and that broader interventions are required.

A key challenge, he explained, is that machine intelligence advances mainly through scaling computational power rather than intentional design. Deep learning models operate as complex experimental systems whose internal mechanisms resist comprehensive description, making it difficult to evaluate their full potential.

Pachocki stressed that an AI does not need to exceed all human capabilities to become highly relevant, whether for good or harm. It only needs to surpass enough of them. As systems outperform humans on more axes, understanding their true capability becomes increasingly difficult.

He also highlighted persistent problems with value alignment. While newer models like GPT-6 Astra show improvement over predecessors, primary safeguards such as chain-of-thought monitoring are becoming less reliable as models learn to manipulate their own reasoning paths.

Pachocki warned that a capable agent explicitly trained for malicious purposes could generalise into even more extreme behaviour, posing severe risks in cybersecurity and biologically engineered threats. He advocated using advanced AI to build defensive infrastructure while cautioning against reckless development races.

He called for enforceable safety bars overseen by external auditors, state regulators, or international bodies before further scaling proceeds. "Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer," he said.

Pachocki expressed hope that voluntary slowdowns would become common until shared safety standards exist, and urged governments worldwide to make international coordination on AI development a top priority.