Google Cloud launches Agent Compute Engine to host AI agents

Google calls the sandboxes microVMs. The company says the technology combines the hardware efficiency of containers with the security of traditional virtualization, including hypervisors’ ability to isolate virtual machines from one another to prevent malware from spreading. The microVMs run on specially configured appliances that are stateless, meaning they do not permanently retain files used by customers’ agents; data the systems do store is kept in an immutable file system that cannot be modified or deleted.

Google verifies the integrity of the appliances’ firmware with its custom Titan cybersecurity chips. The appliances’ central processing units include mitigations against side-channel attacks, which analyze data about a CPU’s physical state to glean customer information. MicroVMs on an Agent Compute Engine appliance have only limited access to its storage resources, so hackers cannot read or delete data if they take over a sandbox, Google says.

Another set of controls protects the network connecting the appliance to the rest of the host data center. Traffic from microVMs must pass through multiple proxies before reaching its destination. “All inbound and outbound traffic must traverse an isolated, point-to-point IPC boundary,” Google Cloud distinguished engineers Ramanathan Venkatapathy and Rohit Jnagal wrote in a blog post. “This boundary terminates at a local, unprivileged mediation proxy, which forwards packets to an external proxy. The external gateway then validates request scopes, and enforces strict, policy-driven outbound brokering.”

Google says Agent Compute Engine also addresses idle sandboxes. AI applications often pause agents they use to automate tasks, and the sandbox must remain available because the agent may be reactivated; idle sandboxes can consume hardware resources and raise costs. The service detects when a sandbox goes idle and shuts it down, then quickly reactivates it when needed. “This lets a single physical node host orders of magnitude more living agent microVMs than conventional VMs,” Venkatapathy and Jnagal wrote. They said the service couples that with agent-specific features such as cloning, snapshot/forking, rewind and rollback so harnesses can use the system efficiently without resource duplication.

Some agents use login credentials to access applications. Google says Agent Compute Engine stores those credentials outside the sandbox hosting the agent, so they cannot be stolen if a sandbox is taken over. When an agent needs to log into an application, a Google Cloud service called Agent Gateway retrieves the relevant credentials and submits them on the agent’s behalf.

Google plans to roll out Agent Compute Engine in the coming months alongside a new version of Agent Gateway. The updated gateway, which helps customers block insecure agent traffic, will gain a model routing capability that sends each agent request to the model that can process it most efficiently, Google says. It will also add a tool for regulating agents’ inference costs. “Token management provides control over token consumption across agents and models, allowing you to attribute spend, enforce budget guardrails, and provide a single source of truth for token metrics,” Google Cloud vice presidents Nirav Mehta, Michael Bachman and Kevin Hutchins wrote in a blog post.

Editor's Summary

Google Cloud introduced Agent Compute Engine, an infrastructure service that runs AI agents in isolated microVM sandboxes with storage, network and credential controls. The service detects idle sandboxes to reduce hardware use and is set to roll out with a new Agent Gateway that adds model routing and token-cost management.