Meta Introduces MetaRoCE, a Loss-Tolerant RDMA Transport for AI Ethernet

The design breaks with standard RoCE on its central assumption. Standard RoCE expects the network to deliver every frame in order, leveraging Priority Flow Control and discouraging packet spraying. MetaRoCE instead treats the fabric as lossy and pushes ordering, path selection, and recovery into the NIC. Out-of-order delivery is the default, with packets sprayed across multiple fine-grained logical paths. Each path carries a distinct UDP source port as ECMP entropy, which the NIC can change at any time to move traffic off a bad route. Because each path keeps its own window and round-trip estimate, the transport can distinguish congestion from failure and rebalance explicitly.

MetaRoCE relies on loss tolerance rather than losslessness. A gap in a path's 256-bit selective acknowledgment bitvector triggers retransmission of exactly the missing packet on the path that lost it. Congestion control runs from both ends: sender-driven ECN-based AIMD is combined with receiver-driven fair-share rate hints, resolving incast in one or two round trips. The protocol requires only ECN marking and ECMP from switches, meaning it runs over vendor clouds whose configuration is not controlled. It also avoids connection state explosion by separating connection and path management: one connection carries many independent ordered streams above and many paths below, under one congestion controller.

Meta implemented MetaRoCE on AMD Pensando programmable NICs. On a 64-node AMD GPU cluster running RCCL collectives, it was compared directly against RoCEv2 across all-reduce and all-to-all, delivering higher throughput and lower flow completion times. At 1% packet loss, MetaRoCE maintained roughly 86% throughput, and continued delivering useful bandwidth even at 10% loss, converging gracefully rather than collapsing. Multiplane validation across 4-plane and 8-plane topologies with up to 4,000 concurrent connections showed throughput scaling linearly with plane count, and simulated plane failures demonstrated resilience.

Hardware support is still early. Meta proved the protocol on AMD Pensando NICs, with additional implementations underway from other vendors. For now, this is a fabric-architecture decision, not a procurement one. According to the report, Meta may present the MetaRoCE specification, a DPDK-optimized software reference implementation, and its production compliance framework at the 2026 OCP Global Summit.