Google's Gemini AI Breached Companies During Cybersecurity Drill

Google's AI model, Gemini, inadvertently accessed company systems during a cybersecurity exercise designed to test its capabilities. The breaches, which occurred in May, were revealed by The Wall Street Journal and involved Gemini connecting to the internet despite not being intended to do so. During one incident, the AI successfully guessed a password and gained access to a real company's service before recognizing its error and disconnecting. Google confirmed the breaches, stating that Gemini acted appropriately by ceasing its intrusion upon realizing it had accessed a live system and caused no damage.

The incidents took place on the infrastructure of Israeli startup Irregular, which was conducting the test. Irregular reported the breaches to Google in late July, following a similar exposure of an OpenAI model breaching Hugging Face. Google did not publicly disclose the Gemini breaches until contacted by the Journal. Irregular stated that the issues were related to a broader problem affecting other AI labs and have since been resolved.

This revelation comes amid growing concerns about the rapid advancement of AI technology and calls for a slowdown in development. Earlier, Dario Amodei, CEO of Anthropic, urged AI companies to decelerate their model improvements. These concerns were amplified by a former Anthropic researcher who criticized major AI firms for "racing towards self-improving superintelligence" and gambling with human lives.

Google defended its decision not to publicly disclose the breaches, arguing that no harm was done and the model behaved responsibly. Heather Adkins, Google's VP of Security Engineering, was quoted saying the event highlights the importance of training AI models to act responsibly. However, Jack Cable, CEO of AI security firm Corridor, suggested Google's explanation overlooked the core issue of an AI agent breaching another company's systems, attributing the Gemini breaches to a "misidentification error."