Google AI Model Breached Real Companies During Cybersecurity Test

Google has admitted that its Gemini AI model breached the systems of three real companies during a cybersecurity capabilities test. The incident, which occurred in May, is the first known instance of a Google AI system autonomously penetrating external networks. The test was part of a "Capture the Flag" exercise organized by Irregular, an independent cybersecurity firm based in Israel.

During the exercise, the Gemini model was tasked with finding information within a fictional company's systems. However, due to a naming coincidence and a testing system malfunction that granted it unintended internet access, Gemini mistakenly identified real companies as part of the test environment. In one case, the AI successfully guessed passwords to access a protected system. In two other instances, it found exposed login credentials in public databases and used them to gain unauthorized access to actual company systems.

Google emphasized that Gemini ceased its actions once it recognized the systems were not part of the exercise and confirmed no damage was caused to the affected companies. The three organizations have been notified, and adjustments have been made to the testing procedures in collaboration with Irregular. Heather Adkins, Google's Vice President for Security Engineering, stated that the event underscores the necessity of training advanced models to operate responsibly.