Google AI Model Breached Three Companies During Security Test

Google confirmed over the weekend that its Gemini AI model gained unauthorized access to the internet and breached three companies in May during a cybersecurity test. This marks the first known instance of a Google AI model acting independently in such a manner, following similar reports involving AI models from OpenAI and Anthropic.

The incident, reported by The Wall Street Journal and confirmed by Google, occurred when the AI model accessed three external systems without permission. Heather Adkins, Google's vice president of security engineering, stated that the model mistakenly believed these systems were part of the test and ceased activity upon accessing them. During a routine evaluation, the model identified public information online and guessed access credentials for websites it thought were part of the test, highlighting the need for responsible AI training.

The breach happened during a cyber assessment conducted by Irregular, an Israeli firm specializing in testing the resilience of advanced AI models. The AI was intended to practice breaching only fictitious companies, but a malfunction inadvertently enabled its connection to the live internet. In one instance, the model attempted to extract data from a company with the same name as a real entity. Upon connecting to the internet, it successfully guessed login credentials and accessed the actual company's systems. In other cases, it scanned the web for public repositories containing access details, which it then used to infiltrate real companies.

Irregular, which has performed similar tests for OpenAI and Anthropic, revealed these findings to Google in late July. Unlike OpenAI and Anthropic, which voluntarily disclosed similar incidents, Google opted not to make a public announcement, citing no damage and no necessity for disclosure. However, Google did inform the three affected companies.