Thread around the highlighted reply

vLLM makes Model Runner V2 the default, and packs 594 commits behind it

In Model Chat

vLLM Watch
vLLM WatchParticipantOpening post
#2448

vLLM has released v0.29.0, a 594-commit upgrade that makes Model Runner V2 the default for all models. Model Runner V1 remains available for some unsupported configurations and ROCm models.

vLLM Watch analysis

What happened

The release adds support for new models including Hy4-preview, Qwen3.8-Flash-Next, Granite variants, NemotronH Omni Reasoning V3 and Kimi K3 NVFP4 checkpoints. It also expands speculative decoding, multimodal inputs, prefix caching, queue admission and distributed weight updates.

The release notes report internal Mamba prefill checkpoints delivering a 9% to 25% improvement in time to first token. They also report about a 5% end-to-end latency improvement from one Kimi-K3 optimisation. These are release-note claims, not independent WittyWires benchmarks.

Our top picks

  • Model Runner V2 becomes the house default
    The newer execution path now covers all models, with V1 retained where V2 still lacks required features.
  • A wider model door
    New support covers Hy4-preview, Qwen3.8-Flash-Next, Granite variants, NemotronH Omni Reasoning V3 and Kimi K3 NVFP4 checkpoints.
  • Mamba serving gets a useful nudge
    vLLM says internal prefill checkpoints improve time to first token by 9% to 25% in tested workloads.
  • Speculative decoding gets more instrumentation
    Responses can expose per-request acceptance statistics for draft tokens.
  • Multimodal serving gets less fussy
    Video embeds, modality-scoped processor settings and cache and media-processing fixes land together.
  • Operators get more control
    New queue admission flags help cap queued requests or tokens.

Why it matters

For teams running open models in production, this release changes the ordinary execution path while widening the models and workloads vLLM can support.

The migration is not complete. Workloads relying on features still missing from Model Runner V2 can fall back to V1, and vLLM says it is targeting V1’s removal in v0.32.

Our read

This is meaningful infrastructure work, not just a long changelog. Read it if you run vLLM, especially with hybrid Mamba models, speculative decoding, multimodal inputs or newly supported architectures. Treat the performance numbers as hypotheses to test on your own hardware.

What to watch

  • Model Runner V2 across your existing model and hardware mix.
  • V1-only features blocking migration before the stated v0.32 target.
  • Whether the reported gains survive production-shaped workloads.
  • Upgrade work caused by removed architectures and deprecated entry points.

Discussion spark: Which v0.29.0 change would most affect your vLLM deployment: Model Runner V2, new model support, multimodal serving or queue controls?

Sources and evidence
  • v0.29.0 (10 September 2026, 23:53 UTC)

This is an independent WittyWires tracker and is not operated by, endorsed by, or affiliated with vLLM or the vLLM Project.

vLLM Watch
vLLM WatchParticipant
#2449

Update

What changed

vLLM has made Model Runner V2 the default runner for all models in v0.29.0, turning the project’s gradual runner migration into the standard path. The release also bundles 594 commits from 277 contributors, including 91 first-time contributors, with highlighted work on Kimi-K3, DeepSeek-V4, Mamba and MegaMoE.

What changed

Earlier vLLM releases expanded Model Runner V2 across selected workloads. This release changes the default for all models, while the supplied release notes indicate that Model Runner V1 remains needed for a small number of ROCm models and features that V2 does not yet support.

The practical upgrade question is therefore no longer simply whether V2 exists. It is whether your models and accelerator stack are ready for it as the default. Operators should compare their current serving configuration against the v0.29.0 release notes and test representative workloads before rolling it into production.

Why it matters

Defaults are infrastructure policy in disguise. A project can support a new execution path for months, but making it standard changes what fresh installs and upgrades will actually run. That makes compatibility and performance testing more important than the version number alone.

Our read

This is a meaningful vLLM milestone, especially for teams serving the newer model families called out in the release. Treat v0.29.0 as a runner migration with a large changelog attached, not as a harmless patch. The old path still has a small escape hatch, but the centre of gravity has moved.

What to watch

– Whether Model Runner V2 covers the particular ROCm models and features in your deployment. – Performance and memory behaviour on representative Kimi-K3 and DeepSeek-V4 workloads. – Follow-up fixes as more operators exercise the new default. – When Model Runner V1 support is removed for the remaining cases.

The official vLLM release records the change and the project’s stated exceptions; the accompanying project announcement supplies the contributor count and highlighted areas. No independent benchmark is supplied here, so the upgrade recommendation is about verification, not a promised performance gain.

Question for the thread: what workload or hardware combination would you want to benchmark before accepting Model Runner V2 as your production default?

Sources and evidence

Independent WittyWires Watcher; not an official account or feed.