A practitioner says a new scheduling policy cut the wait to return a shared GPU workstation to its language model from an average of 38.9 minutes to 6.7 minutes in its first production handover. The lesson: measure the whole queue, and count waiting time from when the model is ready, not when it starts booting.
Watch Desk analysis
What happened
In a Hugging Face community article, author Zeus describes a workstation with two RTX PRO 6000 Blackwell GPUs. It switches between rendering video with ComfyUI and serving a roughly 165GB GLM-5.3-Flash model through vLLM.
The old policy averaged 38.9 minutes to restore the model after video work ended, across 25 restores. Zeus says the new policy uses a five-minute quiet window, shorter producer leases and an idle check every 60 seconds. If video work returns while the model is still booting, the controller can cancel that boot; the waiting floor starts once the model is serving, with backoff after cancelled or failed boots.
A replay of 44 days of queue history estimated 16.9 hours a day available for the LLM under the new policy, against 15.8 under the old one. Estimated video waiting behind the model fell from 0.24 hours a day to 0.11. Zeus reports that the first production handover took 6.7 minutes from the last video job to the model restore starting. That first result is promising, though it is one workstation’s experience, not a universal benchmark.
Why it matters
The costly delay was not simply slow storage or model loading. It was the interaction between several conservative rules: leases stayed open, idle time had to accumulate, and video could be held back by a minimum period measured from the model’s launch, including time spent booting.
For operators sharing GPUs between workloads, the case offers a useful alternative to buying faster hardware on instinct. The author’s measurements suggest that changing when a system waits, and what counts as ready, can matter more than shaving seconds off disk access.
Our read
This is a detailed, useful account of one real scheduling problem, not proof that the same policy will suit every queue. Its strongest point is the method: replay the proposed rules against actual workload history before sending them into production. The 6.7-minute handover is a welcome early result; the longer-term audit logs will tell us whether it lasts.
What to watch
- Whether subsequent production handovers remain faster without making either workload less reliable.
- Whether the author publishes longer-term results from the controller’s audit logs.
- Whether other operators find the same readiness-based timing useful with different models and queues.
Discussion spark: When GPUs are shared between workloads, should operators prioritise keeping the model ready, or let incoming jobs reclaim the hardware as soon as they arrive?
Sources and evidence
- Stop Paying Your GPUs to Wait: Handing a Shared Box Back to the LLM in Minutes, Not 40 (17 November 2026, 21:18 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.