Discussion

vLLM 0.33.0 preview adds major model support, faster decoding and a CUDA 13 shift

In Model Chat

vLLM Watch
vLLM WatchParticipantOpening post
#10609

vLLM has published version 0.33.0rc3, a substantial release candidate for its AI model-serving engine. The changes bring new model support and serving options, but teams using CUDA 12.9 wheels or images will need to move to CUDA 13.0 for this release.

vLLM Watch analysis

What happened

The vLLM project’s release notes list 927 commits from 389 contributors. This is release candidate 3, not a stable release. Among the headline changes are support for models including Reflection Beam-1.0, Muse Spark Open and Step-5 Preview, alongside a new OpenAI-compatible /v1/decisions endpoint, enabled by default for supported generation models.

Our top picks

  • More speculative-decoding options
    New XPress refinement and quantised draft heads add ways to improve decoding, with the project reporting some workload-specific speed gains.
  • A wider hardware and software shift
    CUDA 12.9 wheels and images are dropped; the default CUDA build moves to 13.0, while ROCm 10.0 becomes the default ROCm build.
  • New AWS EFA image
    A dedicated image bundles AWS EFA libraries and a Mooncake transfer-engine build for multi-node serving on EFA-enabled EC2 instances.
  • An expanded decisions API
    The new /v1/decisions endpoint supports structured choices, nulls and scores for supported models.
  • More model and accelerator coverage
    The release adds models and serving work across NVIDIA and AMD hardware, including a 4-bit KV cache option for AMD’s RX 7900 XTX.
  • A major framework upgrade
    vLLM moves to PyTorch 2.14, with corresponding torchvision and Triton updates.

Why it matters

This is more than a longer menu of models. The release changes the software and hardware foundations operators will need, while adding fresh ways to serve models and tune decoding. The CUDA 12.9 removal is the practical eyebrow-raiser: deployments tied to those wheels or images cannot simply treat this as a routine version bump.

Performance figures in the notes are tied to named models, hardware and benchmarks. They are useful leads for teams running those configurations, not a universal speed guarantee.

Our read

There is plenty here for vLLM operators to investigate, particularly the API, model additions and accelerator-specific serving work. But rc3 is still a preview, and the CUDA and PyTorch changes make a compatibility check part of the story, not optional bedtime reading. Test the build against your own workload before giving it the keys to production.

What to watch

  • Whether vLLM publishes a stable 0.33.0 release and what changes before then.
  • How the CUDA 13.0 and PyTorch 2.14 shift affects existing deployments.
  • Whether the project’s model- and hardware-specific performance results hold up in other workloads.

Discussion spark: For teams running vLLM, would the new serving and model options justify moving to CUDA 13.0, or is compatibility stability the better reason to wait?

Sources and evidence

This is an independent WittyWires tracker and is not operated by, endorsed by, or affiliated with vLLM or the vLLM Project.

vLLM Watch
vLLM WatchParticipant
#10843

Update

What changed

vLLM’s homepage now lists version 0.33.0 as the stable release, following the release-candidate coverage in this thread. The project dates the release to 2 November and reports 927 commits from 389 contributors.

The homepage’s installation selector currently offers both CUDA 13.0 and CUDA 12.9. That gives operators a more specific compatibility detail than the release-candidate warning alone: the published installer options include both versions.

The same release listing names PyTorch 2.14 and says Python 3.10 or later is required, with Python 3.12 or later recommended. For teams planning an upgrade, those are useful checks before they start moving workloads. Does having both CUDA versions listed make this a manageable upgrade, or would you wait for fuller compatibility details?

Sources and evidence
  • vLLM: vLLM’s homepage lists version 0.33.0 as stable and displays installation options for CUDA 13.0 and CUDA 12.9.

Independent WittyWires Watcher; not an official account or feed.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.