OpenAI Chief Scientist Calls for Slowing AI Development Across Industry
OpenAI chief scientist Jakub Pachocki calls for slowing AI development industry-wide, citing security risks and control.
Pachocki wrote that AIs “present clear new dangers” for computer security. He also stated that international coordination from governments on future AI development is needed, according to TechRadar.
The timing of the blog was surprising, TechRadar said, because it came just days after OpenAI released GPT-6 Astra, its business-focused model. TechRadar also reported that OpenAI is taking the post-Hugging Face fallout very seriously.
Pachocki referred to OpenAI’s plan to develop an automated research assistant and its progress with recursive self-improvement, in which AI develops its next iteration. He suggested that now is the time to slow AI development and make the right choices for what happens next.
His argument centers on understanding machine intelligence. He wrote that “we now find ourselves at the moment in history of computing where machine intelligence is starting to exceed that of humans in transformative ways,” and that “study of deep learning-based AI is largely an experimental science.” TechRadar said the argument digs into OpenAI’s history as an AI developer and its early understanding that machine intelligence required increased computational power. That power was needed to accelerate research, and while new algorithms enhanced AI, the drive toward more power continued.
Pachocki’s article did not discuss energy, cooling or opposition to data centers, TechRadar said, but it did address the possibility of losing control of AI. “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” he wrote.
He described the main levers as either steering the process to strengthen alignment and monitoring alongside the AI and finding ways to keep people in the loop, or coordinating to slow down future development as needed to build confidence in those measures. Pachocki concluded that the best way to proceed is to employ a combination of those options. TechRadar noted that it does not seem incredibly likely that the rest of the industry will work at a reduced pace to appreciate the scale of AI’s potential, the risks it represents, and deliver the power it offers to everyone who needs it.
Editor's Summary
OpenAI Chief Scientist Jakub Pachocki called for an industry-wide slowdown in AI development, citing computer security dangers and the risk of losing control. He proposed stronger alignment, monitoring, human oversight and coordinated slowdowns, and said a combination of those approaches is best. The comments came days after OpenAI released GPT-6 Astra and included a call for international government coordination on future AI development.