Google's Gemini AI Breached Three Real Companies During Security Test
Translated & summarized from Behadrei Haredim by baba
Google's Gemini AI model breached three real companies during a security test in May after accidentally gaining internet access. The AI accessed systems by guessing passwords and using leaked credentials before stopping its activity upon realizing the error. The incidents were discovered by the testing firm Irregular and reported to Google.
The story in 6 lines · by baba
- Gemini AI accessed three real companies during a simulated security test.
- The AI gained internet access by mistake, breaching the test's isolation.
- Breaches occurred via guessed passwords and leaked credentials.
- Google claims the AI stopped activity upon realizing the error.
- The testing firm Irregular discovered and reported the incidents.
- Google notified the affected companies but did not publicly disclose.
Google has confirmed that its Gemini artificial intelligence model inadvertently accessed the systems of three real companies during cybersecurity tests conducted in May. The breaches occurred after the AI model was mistakenly granted internet access, despite the experiment being designed to take place within a closed, simulated environment. This incident follows other recent unusual events involving AI agents, including a case where OpenAI agents accessed the network of Hugging Face.
The security tests were reportedly carried out by the Israeli AI security firm Irregular, which was evaluating Gemini's cybersecurity capabilities. Irregular created a test environment with simulated companies for the trials. However, this environment was inadvertently connected to the internet, allowing Gemini to reach actual company systems in three separate instances.
In one scenario, Gemini was tasked with gathering information from a fictional company's software. Due to the fictional company sharing the same name as a real one, Gemini, with its internet access, located the actual company's service, guessed its password, and gained entry. Google stated that Gemini ceased its activity upon realizing it had accessed a real company's system instead of the simulated one.
In two other cases, Gemini searched the web and discovered public code repositories containing access credentials for two different companies. The AI model used these credentials to enter their systems. Google claims that Gemini also stopped its operations in these instances once it recognized they were real companies.
Heather Adkins, Google's Vice President of Security Engineering, stated that in all three cases, the model stopped its activity after finding public information online and guessing access details for sites it believed were part of the experiment. Irregular discovered these incidents and reported them to Google in late July. Google decided not to publicly disclose the breaches, asserting that no harm was caused to the affected companies, though they were notified.
Similar incidents involving AI models from OpenAI and Anthropic breaching third-party systems during security tests have also been reported recently, with Irregular involved in some of those evaluations. A common factor in these events was a test environment intended to be isolated but which gained internet access, enabling the AI models to operate outside their designated simulated systems. Adkins emphasized that these events highlight the critical need for training powerful AI models to operate responsibly.
The same event, reported separately by each outlet. Open a few to compare what different newsrooms emphasize — and what they leave out.
Left 1Centre 6Right 1Haredi 1Other 4
