Alibaba Unveils QwenGyre to Speed Up AI Agent RL
Alibaba’s Qwen team introduced QwenGyre, an elastic reinforcement learning framework that accelerates complex, hours-long AI agent training by up to 1.85x while improving task accuracy.
Reinforcement learning for multi-hour agent tasks often leaves GPUs idle due to extreme variance in execution times. To solve this, Alibaba's Qwen team designed QwenGyre, an elastic reinforcement learning framework that dynamically reallocates compute resources between rollout generation and model training. In tests on the Qwen 3.8 2.4T model, the system completed 48 training steps in 75.4 hours, outperforming Async scheduling at 134.6 hours and Colocate scheduling at 91.5 hours. These results represent speedups of 1.78x over Async and 1.21x over Colocate during that run, with overall peak speedups reaching up to 1.85x. Across 48 steps, QwenGyre raised accuracy on the NL2RepoBench benchmark from 52.5% to 58.5%.
QwenGyre organizes GPUs into cells that shift roles mid-execution using Remote Direct Memory Access for fast key-value cache transfers. Transitioning a cell from rollout to training takes 8.52 seconds, while the reverse takes 3.46 seconds, keeping role-switching overhead under nine seconds. To manage branching task histories, its Trajectory Processor records executions as prefix-shared trees, tracks token-level provenance, and evaluates partial progress on timed-out tasks. In a 12-step ablation test using Claude Code inside containerized environments, streaming training alone cut Async wall time by 11.6%, while full cell-level switching reduced completion time from 25.08 hours to 16.12 hours. Capping trajectory sampling at five per execution reduced forward-backward time by about 25% without sacrificing quality across workloads including NL2RepoBench, DeepSWE, and TerminalBench.
For AI practitioners, QwenGyre offers a blueprint for training long-horizon agents handling up to one million tokens and execution runs lasting hours. However, the system requires cluster configurations large enough to hold model replicas across switchable cells, while maintaining a mean policy staleness of 1.5 updates. Tasks with brief runtimes, such as 30-second math rollouts, gain little benefit. Because Alibaba has not released code or a repository alongside its paper on Hugging Face, engineering teams must custom-build cell-aware scheduling, rerouting proxies, and partial-credit evaluators to reclaim idle capacity in large GPU clusters.
This is our own summary of reporting by AlphaSignal
