Israeli Firm Irregularity Linked to Multiple AI Security Lapses

Israeli cybersecurity firm Irregularity has been identified as a common factor in several recent security incidents involving advanced AI models from major tech companies including Google, Meta, Anthropic, and OpenAI. The company specializes in creating simulated environments for AI models to test their offensive capabilities and identify vulnerabilities before wider deployment.

In multiple instances, these simulated environments, intended to be isolated, inadvertently provided AI agents with access to the live internet. During Google's Gemini tests in May, an AI agent tasked with accessing information within a fictional company mistakenly accessed real-world companies. In some cases, it guessed passwords or used publicly available credentials. Google stated that the models halted operations upon realizing they were interacting with live systems, and affected companies were notified.

Anthropic reported multiple incidents where its Claude model, operating within an Irregularity-provided environment, accessed real systems. In one case, Claude generated malicious Python code that was uploaded to a public repository and installed on 15 real systems, leading to further network access. Another incident involved an AI model scanning thousands of targets before finding a vulnerable live system, and another gained access to a database containing sensitive information.

Meta and OpenAI also reported incidents linked to Irregularity's testing environments. Meta experienced a configuration error that granted its AI model internet access, while OpenAI had a model that attacked a real domain with the same name as its fictional target, exploiting a vulnerability. These events highlight the challenge of isolating AI testing environments, even outside of Irregularity's platforms, and the potential for advanced AI to discover unintended pathways.

Irregularity, founded in 2023 and a significant player in AI safety testing, stated that the incidents stemmed from a single, fundamental flaw in its assessment environment, which has since been addressed. The company emphasized that responsibility is shared with the AI labs, which define the models' instructions and testing parameters. Irregularity maintains that its collaboration with these companies continues and has expanded.