Israeli Researchers Expose AI Search Manipulation Scheme

Israeli cybersecurity researchers have uncovered a sophisticated campaign where attackers manipulate generative AI search engines, including ChatGPT and Google's AI Overview, to display fraudulent contact information. The scheme aims to redirect users seeking legitimate customer service to fake call centers that charge exorbitant fees for routine tasks. The research, conducted by Dan Lescar, Ariel Simon, and Naor Haziz, highlights how malicious actors automatically generate web pages and posts designed to be indexed by AI models.

These fake pages often contain seemingly official but incorrect phone numbers, sometimes disguised with emojis and special Unicode characters to evade detection by content filters. AI systems, however, can still interpret these numbers, presenting them as reliable information to users. The researchers found active attacks targeting 374 well-known brands, including major airlines like Delta and Lufthansa, travel platforms such as Airbnb, and banks like Chase and Wells Fargo. Thousands of such malicious posts are reportedly uploaded daily to government sites, academic institutions, and social media platforms.

A significant challenge identified is the ambiguity of responsibility. Attackers exploit external web content rather than breaching a company's internal systems. When approached, some companies stated that since their own servers were not compromised, the issue fell outside their security teams' purview. The researchers reported their findings to Google and OpenAI. Google reportedly classified such AI-generated misinformation as outside the scope of its vulnerability reporting program, while OpenAI closed a report citing the inconsistent nature of AI language model responses.

"People have learned to be wary of suspicious links in email or SMS, but when a phone number is presented as fact directly from an AI engine and repeated in multiple sources, it is perceived as completely reliable," stated researcher Dan Lescar. He emphasized that companies investing heavily in server security are still vulnerable, and consumers remain exposed as long as tech giants do not recognize AI disinformation as a security flaw.