vLLM has published version 0.33.0rc3, a substantial release candidate for its AI model-serving engine. The changes bring new model support and serving options, but teams using CUDA 12.9 wheels or images will need to move to CUDA 13.0 for this release.
vLLM Watch analysis
What happened
The vLLM project’s release notes list 927 commits from 389 contributors. This is release candidate 3, not a stable release. Among the headline changes are support for models including Reflection Beam-1.0, Muse Spark Open and Step-5 Preview, alongside a new OpenAI-compatible /v1/decisions endpoint, enabled by default for supported generation models.
Our top picks
- More speculative-decoding options
New XPress refinement and quantised draft heads add ways to improve decoding, with the project reporting some workload-specific speed gains. - A wider hardware and software shift
CUDA 12.9 wheels and images are dropped; the default CUDA build moves to 13.0, while ROCm 10.0 becomes the default ROCm build. - New AWS EFA image
A dedicated image bundles AWS EFA libraries and a Mooncake transfer-engine build for multi-node serving on EFA-enabled EC2 instances. - An expanded decisions API
The new /v1/decisions endpoint supports structured choices, nulls and scores for supported models. - More model and accelerator coverage
The release adds models and serving work across NVIDIA and AMD hardware, including a 4-bit KV cache option for AMD’s RX 7900 XTX. - A major framework upgrade
vLLM moves to PyTorch 2.14, with corresponding torchvision and Triton updates.
Why it matters
This is more than a longer menu of models. The release changes the software and hardware foundations operators will need, while adding fresh ways to serve models and tune decoding. The CUDA 12.9 removal is the practical eyebrow-raiser: deployments tied to those wheels or images cannot simply treat this as a routine version bump.
Performance figures in the notes are tied to named models, hardware and benchmarks. They are useful leads for teams running those configurations, not a universal speed guarantee.
Our read
There is plenty here for vLLM operators to investigate, particularly the API, model additions and accelerator-specific serving work. But rc3 is still a preview, and the CUDA and PyTorch changes make a compatibility check part of the story, not optional bedtime reading. Test the build against your own workload before giving it the keys to production.
What to watch
- Whether vLLM publishes a stable 0.33.0 release and what changes before then.
- How the CUDA 13.0 and PyTorch 2.14 shift affects existing deployments.
- Whether the project’s model- and hardware-specific performance results hold up in other workloads.
Discussion spark: For teams running vLLM, would the new serving and model options justify moving to CUDA 13.0, or is compatibility stability the better reason to wait?
Sources and evidence
- Releases · vllm-project/vllm · GitHub (Publication date not supplied)
This is an independent WittyWires tracker and is not operated by, endorsed by, or affiliated with vLLM or the vLLM Project.