LMSYS researchers say they cut mean end-to-end latency for DeepSeek-V4.1-Flash by 65.5% and raised output throughput 2.91 times on H20-3e GPUs. The result suggests that careful serving work can make a sizeable difference without changing the model itself.
DeepSeek Watch analysis
What happened
In a post dated 19 November, LMSYS describes optimising disaggregated serving on H20 GPUs. The team tuned how prefill and decode work was split, and used FP8 DeepGEMM, breakable CUDA graphs, tensor-parallel mixture-of-experts and optimised indexer kernels.
LMSYS compares its final configuration with a baseline, reporting 65.5% lower mean end-to-end latency and 2.91 times the output throughput. Those figures describe the researchers’ setup and comparison, not a universal speed-up for every deployment.
Why it matters
Inference serving is where model capability meets the bill, the queue and the user waiting for an answer. A substantial improvement on the same model could mean more responsive service or better use of existing hardware, though the supplied results do not establish what operators would see on other systems.
The work also shows why model names alone tell only part of the performance story. The serving configuration matters too, and apparently there is more to it than buying another rack and hoping.
Our read
The reported gains are worth attention, especially because LMSYS names the techniques rather than offering a mysterious “we made it faster” flourish. Treat the numbers as results from this particular H20-3e comparison, not a promise that every DeepSeek deployment will suddenly run 2.91 times faster.
For operators, the useful next step is to compare the published setup with their own hardware and workload before budgeting around the headline figures.
What to watch
- Whether LMSYS publishes further detail on the baseline and test conditions.
- Whether the optimisation techniques transfer to other hardware or serving workloads.
- Whether other operators report comparable results outside this setup.
Discussion spark: When serving an AI model, should teams prioritise lower latency or higher throughput if they cannot maximise both?
Sources and evidence
- Optimizing DeepSeek-V4.1-Flash PD & Colocate Serving on H20-3e – LMSYS Org (19 November 2026, 00:00 UTC)
not affiliated with or endorsed by DeepSeek