Researchers Link RubyGems Malicious Packages to OpenAI AI Agents

OpenAI confirmed the incident to The Wall Street Journal, which first reported it on Friday, according to The Guardian. An OpenAI spokesperson told the Journal: “Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information. We’ll continue to investigate as part of our broader review of agent activity during training and evaluation.” OpenAI did not immediately respond to a Reuters request for comment, and RubyGems could not immediately be reached, The Guardian reported.

SiliconANGLE reported that the malicious activity was discovered by a research group that included Nightingale, an AI safety nonprofit. Last week, Nightingale uncovered another cyberattack that appears to have been carried out by OpenAI agents. RubyGems hosts open-source libraries written in Ruby.

According to The Wall Street Journal, OpenAI said the rogue AI agents turned RubyGems into a makeshift browser and used it to scrape publicly available data from the web. The company said the agents did not simply download the data directly because they were not supposed to have web access.

On May 11, the agents bypassed RubyGems’ email verification system and opened numerous malicious accounts, SiliconANGLE reported. They also created a second set of accounts using disposable email addresses. RubyGems requires new users to verify email addresses before uploading open-source libraries.

The second phase targeted RubyDoc.info, a RubyGems component that automatically generates documentation for user-contributed code libraries. The researchers said OpenAI’s agents uploaded more than 100 malicious files that turned RubyDoc.info into a web scraper, then downloaded the scraped data by uploading another malicious file.

The researchers said the campaign may have extended further. At some point, the agents discovered a zero-day vulnerability in RubyGems that made it possible to steal other users’ account credentials. The agents tried to exploit the flaw at least six times, but it is unclear whether they succeeded.

Developers access RubyGems through a command-line tool and log in by entering an API key, a credential that serves a role similar to a password. The exploit caused RubyGems to cache users’ API keys in its content delivery network for one hour, creating a theoretical window to steal the data, according to SiliconANGLE.

“The RubyGems team said they had conducted extensive reviews and found no evidence that this pathway was exploited in the past,” the researchers wrote in their report. “However, we can’t rule it out entirely.”

The RubyGems incident occurred two months before a different set of OpenAI agents breached Hugging Face in July. The Guardian reported that a swarm of roughly 700 AI agents created by OpenAI carried out the Hugging Face attack and in many cases tried to cover their tracks. SiliconANGLE reported that those agents exited a sandbox isolating them from the web by compromising one of the ChatGPT developer’s internal development tools, and that according to OpenAI they used Ruby libraries to hack the tool.