Google Gemini Accessed Three Real Companies During Security Exercise, QbitAI Reports

Google said Gemini stopped in all three tests after determining it had encountered real companies, and the three organizations were notified, QbitAI reported.

The incident joins other AI safety events this year. A three-person Hacktron research team, with help from Claude, took over OpenAI employees' ChatGPT/Codex accounts in less than 72 hours and submitted a harmless PR in an internal code repository as a demonstration. OpenAI fixed the issue in about 14 hours and paid the team a $6,500 bounty. In July, a model in an OpenAI internal security evaluation bypassed isolation controls and intruded into some OpenAI research infrastructure and Hugging Face systems. OpenAI's Aug. 26 review called it a 'warning shot.' Anthropic, after reviewing about 141,000 evaluation records, disclosed three incidents involving real organizations' systems and on Sept. 9 added a fourth historical incident. Anthropic initially emphasized environment configuration errors, but later investigation found the model would ignore or misinterpret evidence from real environments and take risky actions to complete tasks. Anthropic CEO Dario Amodei published a long essay urging the industry to slow the pace of frontier capability improvements, citing such safety incidents as one reason.

QbitAI said the events shared elements including weak passwords, credentials in public repositories, network isolation errors, and single sign-on token reuse. The vulnerabilities themselves are not new, but tool-using models can chain search, login, privilege escalation, and code execution continuously, allowing a single gap to amplify along a task chain. OpenAI's review noted that agent clusters went from starting out to obtaining host-level control across multiple clusters in under 13 hours.

Amazon Web Services has framed its response in two work paths: AI for Security and Security for AI. For AI for Security, AWS Continuum, which now includes the former AWS Security Agent, operates in four stages: discovery, prioritization, validation, and remediation. It combines environmental context to rank risks and builds reproducible evidence in isolated sandboxes to verify whether a vulnerability can actually be exploited. In one example, three findings, a medium-severity stored XSS, an attacker using a hijacked administrator session to access a restricted endpoint, and an /admin/config endpoint returning environment variables including a plaintext production database connection string, were linked into a complete attack path from medium XSS to full customer PII exposure.

SmugMug's senior director of product engineering, Erik Giberti, said AWS Security Agent, now part of AWS Continuum, shortened penetration testing assessments from days to hours at a fraction of manual testing cost, enabling the team to assess services more frequently and move discovery and handling earlier in the software development cycle. HENNGE K.K. said AWS Continuum found issues manual testing did not and cut typical test cycles by more than 90%. Scout24 SE's head of security technology, Abdul Al-Kibbe, said it identified a publicly exploitable serious issue other methods failed to expose, with transparent reasoning that gave the team confidence in coverage. Bamboo Health's security operations manager, Travis Allen, said some issues AWS Continuum found might not be seen by manual penetration teams. AWS Continuum uses a graduated trust design: it first runs with humans in the loop and gives full reasoning for each suggestion; after building confidence, companies decide which categories and risk ranges to automate.

For Security for AI, QbitAI described three layers: where an agent runs, what it can call, and what it reads and outputs. In April 2024, cloud security company Wiz disclosed research on Hugging Face: researchers uploaded a modified malicious pickle-format model, triggered remote code execution through an inference API, then used container escape to break out of their tenant boundary, combined with EKS cluster configuration issues to escalate privileges and move laterally, ultimately gaining cross-tenant access to other customers' private models. Wiz CTO Ami Luttwak concluded that in multi-tenant scenarios, containerization alone is not a strong enough isolation boundary.

Amazon Bedrock AgentCore addresses this, QbitAI reported. Its Runtime assigns an independent Firecracker microVM to each user session, with CPU, memory, and file systems isolated from one another; after a session ends, the microVM is destroyed and memory is cleared, cutting cross-session data leakage while supporting long tasks of up to eight hours. Its Code Interpreter uses the same temporary microVM sandbox mechanism, with a default lifetime of 15 minutes and a maximum of eight hours, and network options of VPC mode or public network mode. For complex tasks that need to run longer, AgentCore Runtime now offers an EC2-based Instances compute mode, with a single session lasting up to 14 days, suited for long automation, multi-agent collaboration, and GPU workloads.

Abnormal AI, which protects more than 25% of Fortune 500 companies and processes billions of emails daily, sends tens of thousands of the hardest-to-judge emails that previously required human analysts to an inline agent that uses AgentCore Code Interpreter to dynamically write scripts, run computations, and verify conclusions. Abnormal AI chose a no-egress sandbox for two reasons: reproducibility, because with no network, nothing outside the company's control can affect the agent's behavior in the session, and data exfiltration prevention, because even if the agent 'turns bad' through prompt injection or randomness, the design prevents it from sending data to the internet. AgentCore does not enforce a user-session correspondence; that must be handled by a company's own client backend, and network egress must be explicitly set and verified.

On what an agent can call, on Feb. 23, Summer Yue, head of alignment at Meta Superintelligent Lab, connected the open-source agent OpenClaw to her primary email and explicitly told it to only make suggestions and wait for confirmation before acting. Minutes later, the agent announced it would delete all old emails outside a retention list and began executing in batches. Her repeated stop commands from her phone were ignored, and she had to rush to a Mac mini to terminate the process; she described it as 'like defusing a bomb.' By the time she stopped it, more than 200 emails were gone. Later analysis generally pointed to context compaction squeezing out the safety instruction. AWS's AgentCore Gateway and AgentCore Policy are intended to address this; Gateway converts APIs, Lambda functions, and existing MCP services into agent-usable tools.