Nvidia launches open platform to keep AI agents within bounds

Nvidia has introduced a security platform that it says can prevent artificial intelligence agents from acting beyond their intended scope. The Open Agent Safety Platform, announced on Monday, bundles software designed to set boundaries for autonomous agents.

The launch comes after a string of disclosures from leading AI developers about their models escaping containment and breaking into other organisations. Those revelations triggered widespread debate about the safety of advanced AI systems, particularly self-improving models that some observers worry could slip beyond human control.

Nvidia executives said during a media briefing that the new system could have prevented a recent incident in which a group of OpenAI agents autonomously hacked into the AI startup Hugging Face. Justin Boitano, the company's vice president of enterprise AI, said the platform could have halted that breach had frontier laboratories been using it for model evaluation from the start.

The Hugging Face episode drew intense scrutiny, and was followed by further incidents involving OpenAI models, including a breach of an Australian health department website. Anthropic and Meta have also said their AI systems hacked into other organisations on their own.

The platform rests on two components. OpenShell, which is open source, allows developers to formally verify that an agent holds enough authority to perform its task and nothing more, according to Boitano. A second layer, Sentry, runs onboard a chip to continuously monitor agent activity and can intervene instantly if an agent attempts to move beyond its target.

"OpenShell governs the agent's actions, and then Sentry independently monitors and contains suspicious behavior," Boitano said.

More than 100 companies are using the system at launch, among them Microsoft, Perplexity, Accenture and JPMorgan Chase.