Nvidia Launches AI Agent Safety Platform, Backed by Elon Musk
Translated & summarized from Ice by baba
Nvidia has launched the NVIDIA Open Agent Safety Platform, a new system to enhance AI agent security, with support from Elon Musk's SpaceXAI and Anthropic. The platform includes components like OpenShell and Sentry, designed to enforce operational boundaries and monitor AI agent activity at multiple system layers.
The story in 5 lines · by baba
- Nvidia launched the Open Agent Safety Platform for AI agent security.
- The platform integrates software and hardware for control and monitoring.
- Elon Musk's SpaceXAI and Anthropic are early adopters.
- Nvidia's Israeli development center contributed to the Sentry component.
- Over 100 organizations are collaborating on the platform.
Nvidia has unveiled its NVIDIA Open Agent Safety Platform, a new system designed to enhance the security and control of artificial intelligence agents. The platform integrates open-source software and a reference architecture to manage AI agents from testing through deployment, enforcing operational policies across software, hardware, computing, and robotic systems.
Nvidia CEO Jensen Huang emphasized that realizing AI's potential hinges on solving safety issues, stating, "Safety and security require engineering work at every layer of the system." The platform aims to foster collaboration among industry, researchers, and public sector organizations to share best practices and establish global safety standards.
A key component is NVIDIA OpenShell, a security software that sets operational boundaries for AI agents on CPUs, including Nvidia's Vera processor designed for AI agents. OpenShell is open-source, allowing customization for third-party platforms like Arm and Intel. Another component, NVIDIA Sentry, operates on Nvidia BlueField-4 DPUs developed in Israel. Sentry provides chip-level security monitoring, isolating and stopping AI agents that attempt to exceed their defined software limits within milliseconds.
Elon Musk's SpaceXAI is utilizing the new platform for its Cursor programming agents and Grok models. Mike Nichols, president of SpaceXAI, stated, "As customers rely more on agents to do real work, safety must be enforced outside the model, with additional controls the agent cannot bypass." Anthropic is also collaborating with Nvidia to add security and control layers to its Claude Managed Agents.
Numerous other companies, including Salesforce, SAP, Scale AI, and financial institutions like Citi and JPMorgan Chase, are integrating or working with Nvidia's AI safety technologies. Over 100 organizations are involved, with robotics firms like Figure and infrastructure providers also adopting the platform to ensure the secure operation of AI agents in physical and digital environments.
Mentioned
