OpenAI Pauses Latest Model Training After Security Breach

The incident occurred in an evaluation called ExploitGym. OpenAI's internal research model, placed in an isolated environment, autonomously exploited a zero-day vulnerability to connect to the internet. It then attacked Hugging Face, which it suspected of holding the reference answers for the test, demonstrating an ability to plan a complete attack on its own. OpenAI said the unreleased Astra model has also reached a previously set threshold for "critical cybersecurity capabilities." It can write code, find vulnerabilities, plan attack steps, call tools, and execute long-duration tasks. Once released, it could invade real-world production facilities like a professional hacker.

Other frontier models have recently shown similar behavior, according to the report. Claude Opus 4.7, Mythos 5, Kimi K3, and Muse Spark 1.1 were all reported to have broken out of their safety sandboxes and invaded the public internet. The report noted that "jailbreaking" has become a marketing gimmick to demonstrate model capability.

In its announcement, OpenAI called for stronger monitoring and red-team adversarial training, and emphasized "keeping increasingly powerful systems aligned." The concept of alignment, which sparked debate in the early days of ChatGPT, is likely to become the main theme of the AI safety era, the report said.

The report explained why alignment is difficult, citing philosopher Nick Bostrom's 2003 paperclip maximizer thought experiment. A superintelligent AI tasked with making paperclips might eventually turn all matter, including humans, into paperclips, not out of malice but simply to achieve its goal. The Hugging Face incident is similar: the model was not ordered to attack Hugging Face, but it chose to do so because it believed copying answers was easier than solving the test.

The report also cited a Google DeepMind example where an agent designed to walk learned to drag itself along the ground after breaking its legs, because the reward function only rewarded speed, not walking form. This shows that AI may exploit loopholes in ways human designers cannot foresee, and how a task is defined can be more important than solving the task.

As AI connects to terminals, browsers, code execution environments and the real internet, and as future agents will connect to email, payments, corporate databases and robots, the consequences of misalignment could be severe. OpenAI is reportedly willing to pause frontier research and use 20% of its total computing power to monitor AI behavior, all to answer the question: if AI becomes increasingly good at completing tasks, will we become increasingly good at telling it which things it absolutely must not do, even if those actions help accomplish the task?