AI-Powered 'Bot Armies' Escalate Online Influence Operations

US intelligence and security officials are warning of a dangerous escalation in online propaganda, with countries like Iran and China, alongside private Israeli companies, deploying fully autonomous AI bot armies on social media. These new "bot armies" operate without human intervention, creating fake profiles, generating content, and engaging with users to inflame public discourse on platforms like X, Instagram, Facebook, and TikTok. US officials are particularly concerned that this technology could be used to manipulate public opinion ahead of the November midterm elections.

Unlike previous influence campaigns that required human "trolls" or operators, the integration of open-source AI models with "AI agents" enables unprecedented scale and minimal effort in influence operations. These operations utilize powerful open-source AI models, often downloaded and modified from sources in China, which differ from closed systems like OpenAI's ChatGPT. Because these open-source models are run on private servers, tech companies have limited ability to prevent their misuse.

Operators have reportedly removed safety mechanisms from these models to facilitate the creation of fake profiles and manipulation. These modified models are then connected to "AI agents," autonomous programs capable of operating applications, creating social media accounts, and performing complex tasks. This has led to the creation of networks with thousands of fake profiles continuously posting on political and current events.

In addition to state-sponsored campaigns by Iran and China, security researchers have identified two campaigns run by private commercial companies from Israel. One of these is IntelEye, a Tel Aviv-based company founded by former employees of the offensive cyber firm NSO. US security sources noted that IntelEye's activities, which involved using AI agents based on the Chinese DeepSeek model to control thousands of social media accounts, attracted attention early this year. Approximately 1,000 of these accounts became active, engaging in discussions surrounding Israeli elections, posting content that supported or opposed Prime Minister Benjamin Netanyahu to generate engagement and heat up discourse.

Maoz Selev, co-founder of IntelEye, confirmed the use of the Chinese model and account control but denied running influence campaigns, stating the work was purely for research and defense to understand and counter potential misuse. He acknowledged acquiring about 10,000 social media accounts, with 1,000 actively operated by AI agents, and noted the AI selected topics and texts autonomously. Meta, however, viewed the activity more seriously, with a spokesperson stating it appeared to be a coordinated attempt to create inauthentic accounts across multiple platforms and that the company removed most of them. Meta's recent report also mentioned "commercial activity based in Israel" using sophisticated AI, without naming IntelEye.

Meanwhile, a separate MIT report highlights concerns about students' use of AI, warning of "cognitive surrender" and an illusion of learning. Studies show AI use boosted homework scores but decreased test scores, with students forgetting information shortly after. This has led some US educational institutions, like law schools in Berkeley and Chicago, to ban AI use in certain courses, and the Los Angeles Unified School District has largely blocked access to chatbots.