Israeli Researchers Uncover AI "Poisoning" Campaign Targeting Consumers

Israeli cybersecurity researchers have exposed a widespread "AI poisoning" campaign that manipulates responses from major AI models like ChatGPT, Gemini, and Google AI Overview. The attackers flood the internet with fabricated content, including fake customer service numbers, email addresses, login pages, and software updates, designed to be picked up and presented as factual by AI search tools. This tactic has been observed targeting customers of 374 brands, ranging from banks to airlines.

According to the researchers, the campaign exploits users' trust in AI-generated information, especially in high-pressure situations like flight cancellations or locked bank accounts. When users see a phone number directly from an AI output, they perceive it as highly reliable, unlike suspicious links in emails or texts. The attackers then direct users to fake service centers where they may be charged exorbitant fees, have their login credentials stolen, or lose money.

"Companies invest billions in server security, but this phenomenon proves it's no longer enough," stated Dan Lasker, a cyber vulnerability researcher and co-author of the study. He criticized tech giants for not recognizing AI disinformation as a security vulnerability, leaving consumers unprotected. The researchers developed a system to detect these poisoned AI outputs, noting that the attacks are often statistical and may not be reproducible with every query.

Brands affected include Delta, Lufthansa, United Airlines, American Airlines, Airbnb, TripAdvisor, Chase, Citi, and Bank of America. The malicious content was embedded across various platforms, including social media, blogs, government sites, and university pages. The researchers also noted that the attackers use sophisticated techniques to disguise phone numbers, making them harder for traditional spam filters to detect while remaining understandable to AI models.

Despite the significant damage to consumers and businesses, including lost revenue and reputational harm, companies often deny responsibility, citing that their internal systems were not breached. Both Google and OpenAI have reportedly dismissed the researchers' findings, with Google stating that AI-generated misinformation is outside its bug bounty program's scope, and OpenAI citing the inconsistent nature of the attacks.