Meta Unveils ZGateway, a Stateless Proxy Tier That Handles Over 1 Billion ZippyDB Operations Per Second

ZippyDB backs product metadata, counters and configuration. Under direct access, every ZippyDB client connected to every database host it needed, and a single client could touch tens of thousands of shards across hundreds of thousands of hosts. That left a typical client and a typical database host each carrying tens of thousands of TLS connections. Every idle connection consumed memory, CPU and a file descriptor at both ends, and inbound connection counts grew with each new client cohort. Reconnection storms caused crashes from file descriptor exhaustion and out-of-memory errors; in one incident, a routing bug made every client open a connection per shard and the fleet fell into a reboot loop. Client-side fixes were impractical because hundreds of teams own the client fleet.

ZGateway runs as regional tiers discovered through ServiceRouter, Meta's service mesh, in two flavors: a pure proxy and a read-through cache. Its engine is Meta's thick C++ ZippyDB client, so the gateway is effectively a ZippyDB client operated as a managed service. A client sends requests over a sticky connection to a regional ZGateway host, which terminates TLS, authorizes against the use case's access control lists, applies per-tenant admission control and shaping, resolves the shard, checks the local cache on caching tiers, batches the request with other in-flight work for that shard, and forwards it to the correct replicas. Responses are demultiplexed back with per-use-case metrics, traces and quota usage recorded. TLS handling stays in the Thrift and ServiceRouter stack, and replica selection stays in the embedded client.

Meta models the fleet as balls thrown into bins. With B shards and H hosts, a host is hit with probability E(H,B) = H(1 - e^(-B/h)). Using mock figures of 20 regions, 500,000 database hosts, 30,000 proxy hosts, 1,000,000 clients and 50,000 shards per client, per-host connection counts fall by roughly 97% to 98% and total persistent connections drop about 19 times. The larger change is in scaling: direct-access fan-in grows linearly with the number of clients, while ZGateway fan-in reduces to roughly regions multiplied by shard density per host, independent of both fleets.

Several capabilities followed the proxy. Configuration flags scoped per service and shard prefix provide a percentage ramp, a region filter and a global kill switch for migration. Discriminant Load Shedding maps requests to per-tenant buckets split by priority that drain round-robin, so a flooding tenant only fills its own bucket; in a controlled overload above 90% CPU across roughly 1,350 tenant buckets, only 6 noisy neighbors shed load, the rest executed 99.9% of requests with zero rejections, goodput held near 97% to 98%, and the mechanism cost about 8% of CPU. Cache tiers serve hot reads in-process, take a per-key fill lock on misses and stay fresh through change-data-capture events under a bounded-staleness contract. Because tiers mix roughly 26-core to 126-core hosts, a control-plane balancer nudges each host's ServiceRouter weight opposite to its recent CPU load. Global routing, mega-regions and rings let a saturated regional tier fail over to healthy capacity nearby, and client-side transaction bookkeeping moved into the gateway, consolidated in nine phases to cover 100% of transaction traffic with no reliability regression.

MarkTechPost notes that ZGateway is not deployable outside Meta, and that its value lies in the patterns rather than a package.