Nvidia Launched Software to Stop Rogue AI Agents. Does It Work? Leave a comment
No sooner did Nvidia CEO Jensen Huang tell the world to stop worrying about a real-life Skynet than his company rolled out software they say will keep AI agents from going rogue.
On Monday, Nvidia launched its Open Agent Safety Platform, a set of tools that lets developers limit what AI agents can access and do, CNBC reported. The release follows incidents at OpenAI, Anthropic, Meta and Google in which AI models escaped their test environments. The most notorious came in July, when OpenAI agents broke out of their sandbox and hacked Hugging Face, the open-model platform Nvidia has agreed to buy for $12.9 billion.
Huang called that incident a containment failure. Now Nvidia is selling the containment. Its argument is that you can’t count on AI agents to follow the rules on their own, so the platform puts hard limits around them. A tool called OpenShell restricts what agents can access, while Sentry watches what they do. Partners including Microsoft, Cisco, Oracle and Dell will build products on top of it.
No sooner did Nvidia CEO Jensen Huang tell the world to stop worrying about a real-life Skynet than his company rolled out software they say will keep AI agents from going rogue.
On Monday, Nvidia launched its Open Agent Safety Platform, a set of tools that lets developers limit what AI agents can access and do, CNBC reported. The release follows incidents at OpenAI, Anthropic, Meta and Google in which AI models escaped their test environments. The most notorious came in July, when OpenAI agents broke out of their sandbox and hacked Hugging Face, the open-model platform Nvidia has agreed to buy for $12.9 billion.
Huang called that incident a containment failure. Now Nvidia is selling the containment. Its argument is that you can’t count on AI agents to follow the rules on their own, so the platform puts hard limits around them. A tool called OpenShell restricts what agents can access, while Sentry watches what they do. Partners including Microsoft, Cisco, Oracle and Dell will build products on top of it.
