Israeli Researchers Uncover AI "Poisoning" Campaign Targeting Consumers

Israeli cybersecurity researchers have exposed a widespread "AI poisoning" campaign that manipulates responses from major AI models like ChatGPT, Gemini, and Google AI Overview. The attackers flood the internet with fabricated content, including fake customer service numbers, email addresses, login pages, and software updates, designed to be cited by AI search tools. This tactic aims to trick users into contacting fraudulent service centers, leading to financial loss or identity theft.

The campaign targets customers of 374 brands, including major banks, airlines, and travel platforms. According to researcher Dan Lasker, users often trust AI-generated information, especially phone numbers, as factual, making them vulnerable. The attackers exploit this by creating content around urgent situations like flight cancellations or locked bank accounts, prompting immediate action from users who believe the AI-provided contact details are legitimate.

The researchers, who have backgrounds in classified intelligence units, developed a system to identify these poisoned AI responses. They found tens of thousands of malicious pages, often embedded across various platforms like Instagram, YouTube, government sites, and personal blogs. The attackers use sophisticated techniques, including varying phone number formats and repeating false information across multiple posts, to bypass spam filters and appear credible to AI models.

Despite the significant risks to consumers and businesses, including financial losses and reputational damage, companies are hesitant to take responsibility, often citing that their internal systems were not breached. Both Google and OpenAI have reportedly dismissed the issue, with Google stating that misleading AI-generated information is outside its bug bounty program, and OpenAI claiming the findings were inconsistent and not due to infrastructure breaches.

The research team is now developing a mechanism to counteract these poisoning attacks and restore verified information to AI responses through their new startup. The campaign's automated nature and the persistence of archived malicious content pose ongoing challenges to AI model integrity and user safety.