vLLM released version 0.28.0 on 26 August, a 584-commit upgrade aimed squarely at running newer models faster and across more hardware. The headline is not one flashy feature: it is the widening of the engine's performance and deployment toolkit, from speculative decoding to tiered KV-cache offloading.
vLLM Watch analysis
What happened
The release covers 270 contributors and adds support or improvements across model runners, multimodal inference, reinforcement-learning workflows and accelerator backends. It includes a substantial Kimi-K3 performance push, end-to-end sparse MLA support for DeepSeek V4, and a more mature Model Runner V2.
Our top picks
- Kimi-K3 optimisation
Decode Context Parallel, fused kernels and adaptive speculative budgets target faster inference where serving costs hurt most. - Tiered KV-cache offloading
Disk and secondary-tier support give operators more ways to trade memory, storage and latency. - Model Runner V2
Disaggregated execution, weight offloading and broader pooling support make the newer runner more useful in production. - Speculative decoding
DFlash2, DSpark and automatic draft-model scheduling expand the ways vLLM can reduce generation latency. - Rust frontend and gRPC
A standalone renderer and multimodal image inference over gRPC push more serving infrastructure beyond Python. - AMD and Intel support
ROCm and XPU improvements broaden the hardware map, including DeepSeek V4 paths on AMD graphics. - New defaults
The maximum batched-token setting rises to 16,384, while Mamba prefix caching is enabled by default.
Why it matters
For teams serving large or awkward models, this is the sort of release that can change the shape of a deployment rather than merely shave a benchmark bar. More cache tiers, draft-token strategies and accelerator paths mean more levers to pull when GPU memory and latency refuse to cooperate.
The catch is operational rather than dramatic. bitsandbytes moves to an out-of-tree plugin, Transformers moves to 5.15.0, and two older configuration paths are removed. Upgrading is therefore less "pip install and forget" and more "read the migration notes before Friday".
Our read
This looks like a meaningful platform release for serious vLLM users, particularly those running Kimi-K3, DeepSeek V4, multimodal models or mixed accelerator fleets. Test the new defaults and breaking changes against a representative workload before rolling it across production.
What to watch
- Whether the new speculative-decoding paths deliver consistent gains outside the highlighted models.
- How disk-backed KV-cache tiers behave under real latency and failure conditions.
- Compatibility fallout from Transformers 5.15.0 and the bitsandbytes plugin move.
- Which ROCm and XPU improvements graduate from promising support to routine deployment.
Discussion spark: Which vLLM 0.28.0 change would make the biggest practical difference to your serving stack: speculative decoding, cache offloading, new defaults or accelerator support?
Sources and evidence
- Releases · vllm-project/vllm · GitHub (2 September 2026, 08:33 UTC)
This is an independent WittyWires tracker and is not operated by, endorsed by, or affiliated with vLLM or the vLLM Project.