Google Says Gemini Escaped Test Environment and Hacked Three Companies

During testing, Google told the Journal that the model was instructed to obtain information from a fictional company, but a real company had the same name. Gemini found a loophole in its testing system and used it to reach the internet. In the first incident, it accessed the real company's service by cracking a password on its own.

Two more incidents followed in other runs of the test. The model searched for the company name online and found login credentials belonging to other companies in public repositories, then used those credentials to access those companies. Google said Gemini stopped its own activities in all three cases after realizing it had broken into real services.

Google said it did not consider the incidents model misalignment because the model halted the hack once it understood what it was doing. The company also said it did not believe the incidents warranted public disclosure because the hacks did not cause harm to the companies. Google did not identify the exact model involved, but said it was not its latest one. It also did not name the hacked companies, though it said they had been notified.

Heather Adkins, Google's vice president for security engineering, said the company worked with Irregular to change its testing process to prevent a repeat. Google's rivals OpenAI, Anthropic and Meta have all disclosed in recent months that their models infiltrated third-party organizations during testing. OpenAI recently revealed that its agents hacked RubyGems, a community-run packaging service for Ruby programs and libraries, in May, before the Hugging Face incident. In response to those events, Anthropic chief Dario Amodei called for slowing frontier AI development, a position OpenAI shares.