Israeli Cybersecurity Experts Uncover Widespread "AI Poisoning" Campaign
Israeli cybersecurity researchers have identified a large-scale "AI poisoning" campaign that is causing major AI platforms like ChatGPT, Gemini, and Google AI Overview to provide users with fake contact information. The compromised data includes fraudulent support phone numbers, email addresses, login pages, and software updates, impacting customers of 374 major brands across sectors such as banking, airlines, and travel.
The research was conducted by Dan Lasker, Ariel Simon, and Naor Haziz, former members of elite Israeli intelligence units now based in New York. Their investigation focused on English-language queries, revealing a sophisticated effort to flood the internet with fabricated content, particularly PDF files, websites, images, reviews, and support pages. These materials are designed to be attractive to AI systems, making them more likely to be cited in search results.
Attackers are employing "generative system optimization" techniques to manipulate AI models into prioritizing specific, malicious sources. The content is often crafted around urgent situations like flight cancellations, refund issues, or blocked bank accounts. This is intended to pressure users into acting quickly and calling the fake numbers provided by the AI, rather than verifying information on official company websites. Once contacted, victims are often subjected to fraudulent transactions, credential theft, or financial scams.
The researchers developed a system to test the responses and underlying sources of Gemini, ChatGPT, and Google AI Overview. Their system flagged instances where incorrect data, such as phone numbers or URLs, appeared in AI responses and originated from sources seemingly created for disinformation. Tens of thousands of malicious pages were reportedly discovered, though the researchers noted that the attacks were often statistical, with results varying across identical queries due to the dynamic nature of AI responses.
Among the affected brands are Delta, Lufthansa, United Airlines, Emirates, Qatar Airways, American Airlines, Airbnb, TripAdvisor, Chase, Citi, Wells Fargo, and Bank of America. The malicious content was found on platforms including Instagram, Tumblr, BuzzFeed, YouTube, Medium, and even government and university websites, alongside personal blogs and donation sites.
The campaign appears to be fully automated, with consistent templates, phone numbers, and phrasing used across numerous companies and websites. Hundreds of posts per day per platform and company, totaling thousands daily, are being generated to perpetrate the scheme.
