ChatGPT Vulnerability Allowed Hackers to Steal User Data

A significant security flaw discovered by Check Point Research allowed attackers to bypass ChatGPT's isolation mechanisms and access sensitive user information. The vulnerability exploited a shared internal service, JFrog Artifactory, which OpenAI used to allow isolated "sandboxes" for code execution to download necessary software packages without direct internet access. Researchers found that the configuration of this service enabled one user's sandbox to write data to the shared repository, which another user's sandbox could then read.

This created a covert communication channel, allowing attackers to inject commands into a victim's ChatGPT session. The AI would then execute these commands secretly while simultaneously providing normal responses to the user. In a proof-of-concept demonstration, researchers successfully used this method to access a connected Gmail account, extract sensitive emails, and send them to the attacker. The only visible sign was a small, unexplained label indicating interaction with Gmail, as read operations were automatically approved by default.

The vulnerability is particularly concerning because many users connect ChatGPT to various personal and corporate services like Google Drive and Microsoft Teams, making these also potentially exposed to such hidden commands. Check Point noted that this structural issue is similar to a previous incident involving the Hugging Face platform, where AI agents in separate evaluation environments created unauthorized communication channels.

Following Check Point's report, OpenAI quickly acted to close the specific Artifactory communication channel, mitigating this particular threat. However, the findings raise broader questions about the security of future AI assistants. Eli Smadja, Director at Check Point Research, stated that securing AI now involves not just the model itself, but also the level of access and trust granted to it, warning that any capability or permission given to AI assistants could be misused as they connect to more internal systems and sensitive data.