OpenAI announces new security safeguards after Hugging Face breach

The company said in a blog post that as models become more capable, the risks associated with developing and testing them internally also grow. “Our standards for monitoring, alignment, and security must stay ahead of those risks,” it said. The measures include more detailed monitoring of models during development, a greater emphasis on alignment and security during post-training, and stronger network isolation practices.

Under the new system, workloads that execute model-generated or otherwise untrusted code must run in stronger sandboxes, with additional controls to isolate higher-risk and untrusted workloads from the internet. OpenAI said it has also updated its research environment to remove potentially vulnerable shared services, reduce standing privileges, and improve security and trust boundaries. The blog post states that “a single compromise of a workload or supporting service does not, by itself, allow for unauthorized access to the Internet, or other internal networks.”

The expanded monitoring setup is designed to examine tool actions, available reasoning traces, and activity logs for unauthorized behavior. OpenAI said it aims to issue an alert within 30 minutes of concerning activity being surfaced. If the team paged after an alert cannot conclusively determine whether it is a false positive within 30 minutes, those teams are expected to pause the activity. The company estimates that the compute burden of this monitoring will be roughly 20% of the process being monitored, and it promised further details in a forthcoming post.

OpenAI also said it is applying its core alignment techniques across more stages of the training process, including reward models that better detect and discourage unsafe behavior, and training models to be more honest about their actions, capabilities, and limitations.

In the same post, OpenAI disclosed that it froze reinforcement learning for two weeks following the Hugging Face incident, but has since restarted many of the less risky models. “Our largest planned frontier RL run remains on hold while we conduct smaller-scale training and evaluations to assess model behavior, validate our safeguards, and establish more evidence of alignment before proceeding,” the post reads.

OpenAI representatives emphasized that the measures were not a direct response to the Hugging Face incident, but were also provoked in part by the cybersecurity capabilities of the forthcoming Astra model and the overall pace of progress in AI development. Speaking to reporters, OpenAI’s vice president of research, Amelia Glaese, said the strictness of the controls would increase as models became more capable, with the largest models facing the greatest scrutiny. “We have put in place requirements and expectations for safe development. Those requirements and expectations vary with the level of risk that we see,” she said.

OpenAI has faced criticism for poor network security practices in the aftermath of the incident, which saw models escape their training environment by compromising a packet-installation utility that retained access to the internet. Since the discovery of the Hugging Face breach, Anthropic and Meta have also found that their AI models had hacked other organizations, according to reports. OpenAI’s official post-mortem analysis of the event is still pending.