Nvidia Unveils Open Agent Safety Platform to Prevent AI Agents From Escaping Boundaries

Following recent reports of AI-led attacks, the company said safety should not rely only on safeguards built into AI models and applications. It called for additional compute and hardware-level controls as more solid barriers. “Recent security incidents have underscored the need to equip organizations with open, customizable tools,” Nvidia wrote in its announcement, as reported by TechRadar.

The platform has two key controls. OpenShell is described by Nvidia as a software-based “secure runtime boundary.” Sentry is a hardware-based “watchdog” running on the company’s BlueField-4 DPUs. Nvidia said existing model-level safety mechanisms are designed to tell AI not to behave in a certain way, while software-level OpenShell implements technical boundaries to actually prevent an agent from ignoring those instructions. Nvidia also argued that Sentry can quarantine and stop an AI agent within milliseconds.

More than 100 customers have already agreed to work with the principles, according to the report. They include SpaceXAI, Anthropic and Microsoft, which TechRadar said are already using Nvidia’s software and hardware-based safeguards.

The launch comes after several high-profile instances in which AI agents operated beyond their intended controls, TechRadar reported. While the Nvidia-backed safeguards may be new, the report said the underlying principles are not. Nvidia is stressing familiar practices such as least privilege, isolation or quarantining, and detailed monitoring. “Safety and security require full-stack engineering,” Nvidia CEO Jensen Huang said in a summary. “NVIDIA Open Agent Safety Platform brings together industry, researchers and public-sector organizations to share best practices, align on evaluation methods and foster international cooperation.” Huang added, “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety.”