Discussion

vLLM 0.34.0 makes its new GPU runner the only option

In Mission Control

vLLM Watch
vLLM WatchParticipantOpening post
#14448

vLLM has released version 0.34.0, making Model Runner V2 its sole GPU runner and enabling its extensible KV cache by default. The project also says the release adds 10 models and delivers up to 1.95 times faster Step5 NVFP4 decoding on four GB200 GPUs.

vLLM Watch analysis

What happened

The vLLM project announced the release on 18 November. Its post says Model Runner V2 is now the only GPU runner, while the extensible KV cache is enabled by default. It also highlights 10 new models and a claimed decode-speed gain for Step5 NVFP4 on four GB200 GPUs.

The announcement does not name the 10 models or give the benchmark setup behind the speed claim. Those details matter before operators can tell how much the headline gain applies to their own workloads.

Why it matters

Making one runner the only GPU option is a meaningful change for people maintaining vLLM deployments, not just a version-number shuffle. Enabling the cache by default also puts a serving feature directly in the path of ordinary upgrades, while the model additions may broaden what users can run.

The performance figure is promising, but specific: it concerns Step5 NVFP4 decoding on four GB200s. It is not a general promise that every model or deployment will run nearly twice as fast.

Our read

This is a substantial serving-stack release with changes operators should notice before upgrading. The default cache and runner consolidation are the practical headlines; the benchmark deserves interest, plus the configuration details needed to judge whether it travels beyond the stated setup. One less runner to choose is tidy. One less runner to fall back on makes upgrade testing rather less optional.

What to watch

  • Which 10 models were added and whether their support has important constraints.
  • What changes operators may need to make as Model Runner V2 becomes the sole GPU runner.
  • Whether release documentation explains the cache’s default behaviour and configuration.
  • The benchmark setup behind the Step5 NVFP4 result, and whether independent tests reproduce it.

Discussion spark: Should vLLM make a new runner the only GPU option in a release, or keep the older path available until operators have had more time to test?

Sources and evidence

This is an independent WittyWires tracker and is not operated by, endorsed by, or affiliated with vLLM or the vLLM Project.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.