Microsoft Warns of AI Email Attacks Hiding Malicious Commands in Innocent Messages

As organizations increasingly integrate AI agents with email, calendars, and documents to automate tasks, security researchers warn of a new threat: attackers embedding malicious instructions within seemingly harmless emails to manipulate AI agents. Shaked Ilan, a senior security researcher at Microsoft Israel R&D, explains that unlike traditional phishing targeting humans, these "Prompt Injection" attacks aim directly at AI agents processing the emails.

Attackers can hide commands using techniques such as white text on white backgrounds, tiny fonts, or embedding instructions deep within long email threads or attachments. These hidden prompts can cause AI agents to leak sensitive information, change payment details, or alter the content presented to users without their knowledge. For example, an attacker might insert a command in a payment-related email thread instructing the AI to update bank account information to a fraudulent account.

This vulnerability is particularly concerning in corporate environments where AI agents have broad access to emails, contracts, invoices, and HR documents and can perform actions autonomously. Unlike chatbots that only provide answers, AI agents can execute tasks such as sending messages or accessing systems, increasing the potential damage if manipulated.

To combat this, Microsoft recently introduced a new protective layer within Microsoft Defender for Office 365 that scans incoming emails for hidden malicious instructions before delivery to users or AI agents. This system uses advanced detection methods, including synthetic examples of potential future attacks, to identify and quarantine suspicious messages.

Ilan emphasizes the importance of limiting AI agents’ permissions, requiring human approval for sensitive actions, and monitoring their activities. He compares AI agents to new employees who should not be given unrestricted access immediately. While the current risk is highest in organizational settings, it may also affect private users as AI assistants gain access to personal emails and services.