Cyber Experts Uncover "Dark Sourcery" AI Manipulation Scheme

Israeli cybersecurity experts have uncovered a large-scale scheme, dubbed "Dark Sourcery," where malicious actors manipulate AI chatbots like ChatGPT and Google Gemini to display fraudulent contact information. The attackers embed optimized content, including fake support pages and reviews, across the internet. This content is then picked up by AI systems, leading them to present fake phone numbers, email addresses, and phishing links as legitimate contact details for numerous brands. Researchers identified attacks targeting 374 well-known brands, including major airlines like Delta, Lufthansa, United Airlines, Emirates, and Qatar Airways, as well as travel platforms such as Airbnb and TripAdvisor, and financial institutions like Chase, Citi, and Wells Fargo. For instance, a user seeking to rebook a flight or contact a bank might receive a fake call center number from an AI. Operators at these fraudulent centers then attempt to steal banking information or charge exorbitant fees. The researchers emphasize that this is not a hack of the AI models themselves, but rather an "information poisoning" of the data sources they rely on. They reported their findings to Google and OpenAI. Google stated that such disinformation falls outside its vulnerability disclosure program, while OpenAI closed the report, citing the difficulty in consistently reproducing the results due to the dynamic nature of AI responses. The researchers advise users to always double-check sensitive information, such as support numbers or payment links, on official company websites.