Israeli Researchers Uncover New AI-Driven Scam Targeting Consumers
Israeli cybersecurity researchers have identified a novel scam method that exploits public trust in artificial intelligence (AI) tools, potentially leading to financial fraud. The study, conducted by Dan Lasker, Ariel Simon, and Naor Haziz, former elite IDF intelligence unit members, found that a significant majority of users (92%) accept AI-generated information as fact without verification.
Attackers are leveraging this blind trust through automated campaigns called GEO, or Generative Engine Optimization. This technique aims to manipulate the information presented by AI engines like ChatGPT, Gemini, and Google AI Overview. Instead of direct spam, hackers inject seemingly credible data, such as phone numbers embedded with special Unicode characters and emojis. AI systems may misinterpret these as official contact numbers.
The scam scenario involves a user seeking legitimate information, like a bank's operating hours or flight cancellation procedures. The AI then displays a seemingly official phone number. When the user calls, they reach a fraudulent call center impersonating the legitimate service, which may offer services for exorbitant fees.
The researchers' system has detected active attacks targeting 374 international brands, including major airlines like Delta, Lufthansa, United, Emirates, and Qatar Airways, as well as travel platforms like Airbnb and TripAdvisor, and banks such as Chase, Citi, and Wells Fargo. The problem is exacerbated by malicious posts appearing daily on government, academic (.edu), and social media sites. Even after removal, cached versions can persist and continue to feed AI models.
The report highlights a responsibility gap. Attacked companies argue they are not liable since their internal servers were not breached. Tech giants, however, are also evading responsibility. Google considers AI-generated disinformation outside its bug bounty program's scope, while OpenAI reportedly closed a report, citing the inconsistent nature of AI responses. Researcher Dan Lasker stated, "People have learned to be wary of suspicious links in emails or texts, but when a phone number is presented as fact directly from an AI engine and repeated in several sources, it is perceived as completely reliable." He added that until tech giants acknowledge AI disinformation as a security vulnerability, consumers remain unprotected.
