Alibaba's Qwen Unveils QwenGyre, an Elastic Reinforcement Learning Framework for Long-Horizon Coding Agents

The team defines the target workload as extreme-long-horizon tasks, in which an agent run spans hours and hundreds of interactions with models and tools. Generating a complete repository from a specification, as measured by NL2Repo-Bench, is one example. Two problems emerge at that scale. Rollouts have long tails, so in a colocated setup training waits for the slowest execution while GPUs sit idle, and in an asynchronous setup fixed GPU pools cannot lend spare capacity to one another. Separately, harnesses such as Claude Code compact context, spawn sub-agents and retry, turning a single execution into a branching tree rather than one trajectory; expanding every path duplicates shared prefixes and inflates training cost.

QwenGyre addresses the first problem with an elastic scheduler that splits GPUs into cells sharing a training layout. Each cell can either serve rollouts or host a full training replica. A core cell holds the authoritative weights and optimizer state, while satellite cells join training later. The scheduler tracks a rollout waterlevel of queued and running executions, and as executions finish, cells switch to training once the remaining cells can still serve all live work. Model calls are rerouted through a proxy so the harness keeps running, with KV-cache blocks migrating over RDMA during rerouting to reduce prefix recomputation. Satellites pull core weights through Mooncake's Transfer Engine and join an ongoing batch, and streaming training lets cells claim micro-steps as data becomes ready. Average role switches took 8.52 seconds from rollout to training and 3.46 seconds from training to rollout.

A trajectory processor builds training samples from the branching histories. A proxy records exact token IDs and behavior log-probabilities using token-in, token-out, and calls are stored as prefix-shared trajectory trees. Timed-out executions are still scored on their preserved workspace, while evaluator failures are excluded rather than treated as zero reward. At most five trajectories are kept per execution, ranked main, main-summary, sub-agent and sub-agent-summary; shared targets count once and only policy-generated tokens are trained. Losses average within each execution so extra branches do not increase its weight. The team reports that the cap of five matched uncapped training scores while cutting forward-backward time by 25.2 percent. Training uses an adapted GSPO with 16 rollouts per group.

In benchmark runs that used Claude Code 2.1.220 as the harness with equal GPU budgets and matched staleness, QwenGyre was 1.38x to 1.78x faster than an asynchronous baseline and 1.21x to 1.85x faster than a colocated baseline at matched scores. On NL2RepoBench with Qwen 3.6 122B over 48 steps it reached 1.42x to 1.53x over Async and 1.36x to 1.47x over Colocate; on DeepSWE over 24 steps, 1.569x to 1.572x and 1.610x to 1.824x; and on TerminalBench over 24 steps, 1.383x to 1.430x and 1.809x to 1.849x, the latter being the best end-to-end result. The 122B runs used 24 to 32 nodes and the 2.4T run used 192 nodes arranged as four cells of 48. Megatron handles training and SGLang serves rollouts; the GPU model was not disclosed.

On the 2.4T model, QwenGyre completed 48 steps of NL2RepoBench in 75.42 hours, against 134.55 hours for Async and 91.47 hours for Colocate, an evaluation pass rate of 52.48 percent rising to 58.54 percent. That run produced the smallest margin, 1.21x over Colocate, and the report attributes it to many queries running near the timeout. The paper links no public code release, and the license is not disclosed.

QwenGyre enters a field that includes ByteDance Seed's veRL, Tsinghua IIIS and Ant Group's AReaL, and Microsoft's Agent Lightning, which are released under Apache-2.0, Apache-2.0 and MIT licenses respectively. AReaL states a 2.77x speedup over a synchronous version 0.3 baseline, while the other headline figures come from each project's own setup.