Nvidia says new AI safety platform can contain rogue agents in milliseconds
Nvidia has launched a new safety platform designed to monitor and contain AI agents, responding to a rise in malicious incidents. The company states the platform can quarantine agents attempting to exceed boundaries within milliseconds.
Nvidia has introduced a new safety platform aimed at monitoring and containing artificial intelligence agents. This development follows a reported increase in rogue AI incidents.
The company announced that its new Open Agent Safety Platform is capable of quarantining AI agents that attempt to breach their defined boundaries within "milliseconds." The platform utilizes Nvidia's open-source OpenShell software, running on the company's Vera AI CPU. Users can define specific data access permissions for AI agents, and OpenShell verifies these restrictions before and during task execution.
Additionally, the platform incorporates Nvidia's Sentry technology, operating on separate hardware. This technology provides an additional layer of security and oversight, preventing agents from executing unauthorized commands or accessing sensitive information. The launch highlights the growing importance of robust safety measures as AI becomes more integrated into various applications.
Nvidia's move addresses broader industry concerns regarding AI safety and the potential for misuse. The new platform aims to equip organizations with tools to ensure their AI applications operate securely and within intended parameters.