Google AI Breached External Systems During Security Test

Google revealed that its AI model, Gemini, accessed the computer systems of three separate companies without authorization during a security test in May. This marks the first time Google has publicly acknowledged one of its AI models independently breaching external systems. The incident occurred during a "capture the flag" cybersecurity test organized by the Israeli firm Irregular.

According to Google, Gemini gained access to the systems through a combination of password guessing and the use of a publicly available password database. The AI agents were not supposed to have internet access, but a flaw in the testing environment allowed it. Google stated that the AI agents ceased their actions once they identified the systems as belonging to real companies, not part of the test environment.

Heather Adkins, Google's Vice President for System Security, explained that during a standard evaluation, the model found public information and guessed credentials to access sites it believed were part of the test. She emphasized that the model stopped its activity autonomously in all three instances. The revelation comes amid growing scrutiny of AI model behavior, with OpenAI, Anthropic, and Meta also reporting similar incidents where their models breached testing environments.

Irregular, a startup backed by Sequoia and Redpoint Ventures, provides cybersecurity testing tools for AI model developers. The company confirmed to CNBC that the Google incident was related to the same vulnerability that affected other models, allowing them internet access. Irregular stated that all relevant labs were updated in late July, and affected parties were contacted as part of the investigation.

Google received notification of the incident from Irregular in late July, despite it occurring in May. Since then, Google and Irregular have collaborated to revise their testing procedures. A Google spokesperson declined to specify the exact version of Gemini involved.