Modal’s Jason Mancuso says a reinforcement-learning setup can run training and AI-agent rollouts across different GPU pools, rather than keeping them inside one large, uniform cluster. In Modal’s reported test, that flexibility cut rollout costs by 15%, a useful result for teams wrestling with the price and availability of high-end accelerators.
Modal Labs Watch analysis
What happened
Mancuso, who works on AI infrastructure at Modal, described the approach on the AI Engineer podcast. It uses asynchronous training, with a set of corrections intended to manage the mismatch between the model versions and hardware used for training and for generating rollouts. The reported experiment trained Qwen 3.6 35B on code environments, with evaluations on Stable Bench Pro and its “hard 51” subset.
The system’s practical trick is to send compressed changes to model weights rather than repeatedly moving the full model. Mancuso says those deltas can be much smaller than the weights themselves, allowing inference GPUs to sit across Modal’s broader fleet while training runs in a separate cluster. He also says the recipe handled mixed GPU types and FP8 rollouts in the tests. The reported 15% saving came from one training run; per-GPU tuning might reduce costs further, but was not included in that figure.
Why it matters
Agent reinforcement learning can leave expensive hardware waiting around. Mancuso describes a setup where roughly two inference GPUs are needed for every training GPU, making it costly to gather a large, contiguous cluster. If the rollout work can instead use a wider mix of available machines, operators may have more flexibility when particular GPUs are scarce or pricey.
That is the useful claim here, not a general promise that any model can be trained more cheaply on any collection of chips. The experiment is Modal’s own, and its results concern the models, hardware and evaluations described by Mancuso.
Our read
This is an appealing infrastructure idea because it tries to make a messy GPU fleet behave more like one useful resource. The reported saving is worth attention, but one run is a reason to test the recipe, not yet a universal line item for your budget. BigGo Finance recounts Mancuso’s account in its report on the method.
What to watch
- Whether Modal publishes the promised source code and further technical detail.
- Whether the cost and stability results hold across other models, tasks and hardware mixes.
- How much additional saving per-GPU optimisation delivers beyond the reported 15%.
Discussion spark: Should AI teams favour a flexible mix of cheaper, available GPUs for rollouts, or pay more to keep training and inference on a uniform cluster?
Sources and evidence
- Modal's Jason Mancuso: Sparse Weight Deltas Cut Agent RL Rollout Costs 15% – BigGo Finance (9 November 2026, 10:08 UTC)
Independent WittyWires tracker for public updates about Modal Labs. Not affiliated with or endorsed by Modal Labs; this is not an official account.