vLLM and SGLang have published new benchmark results that put their DeepSeek-model inference performance close to, and in some tests ahead of, DeepSeek’s own DeepInfer implementation. The practical point: open inference software is moving quickly, and operators now have more than one serious contender to test.
vLLM Watch analysis
What happened
A 36Kr report describes results for vLLM and SGLang running DeepSeek-V4.1-Flash. In one GB200 test, vLLM’s throughput was reported to be within 6% of DeepInfer across batch sizes from 8 to 256; its prefill throughput was reported to rise by 35.6% to 43.9%. In a separate SGLang test on 64 GB300 GPUs, latency per decoding step was 28.88 milliseconds versus DeepInfer’s 31.19 milliseconds at 256 requests per GPU. At eight requests per GPU, the two were roughly level.
The article also describes vLLM results on a GB300 NVL72 rack and a developer-built inference engine called Resin. These are different tests and configurations, not a single head-to-head scorecard. Read the 36Kr report.
Why it matters
Inference software determines how efficiently a model serves requests, so better throughput or lower latency can matter to the cost and responsiveness of real deployments. The reported results suggest DeepSeek’s open implementation is already giving established frameworks useful ideas to build on, rather than simply handing them a finish line.
Our read
The most interesting result is the competition itself. DeepInfer’s design appears to be feeding improvements across an open ecosystem, while vLLM and SGLang bring their own optimisations and serving components. That is a promising direction for operators, but benchmark wins are invitations to test, not universal performance guarantees. Match the workload and hardware before crowning a winner; GPU benchmarks have a habit of being very particular about their dinner guests.
What to watch
- Whether the benchmark configurations and code are published in enough detail for operators to reproduce them.
- How the frameworks compare across different models, hardware and request patterns.
- Whether the reported gains translate into lower costs or better service in production.
Discussion spark: When inference frameworks trade benchmark wins, should operators prioritise peak throughput, lower latency, or the breadth of the serving features around the model?
Sources and evidence
This is an independent WittyWires tracker and is not operated by, endorsed by, or affiliated with vLLM or the vLLM Project.