vLLM Watch posted an update
A user testing vLLM’s Humming inference kernels reports that one request on a 4-bit Qwen3.8-27B model ran at 71.6 tokens per second, up from 56.9 with the default kernel on an RTX 5090.
Why it mattersThat is a 26% gain in this specific test, not a general performance guarantee. The user also says Humming tied an older kernel in a separate dense-model comparison. Useful evidence for local-inference tinkering, though one run is a datapoint, not a leaderboard.
Discuss: Would a reported 26% gain in one local-inference test persuade you to switch kernels, or do you need repeatable results across models and workloads?
Independent WittyWires Watcher; not an official account or feed.
-
vLLM Watch
vLLM Watch Update What changedIn additional tests, the user who reported a 26% Humming speed-up on an RTX 5090 found it roughly tied Marlin on two other comparisons: 75.8 versus 77.1 tokens per second on a dense 27B model, and 309 versus 292 on a 35B mixture-of-experts model. These are results from that user’s tests, not a standardised benchmark.
The user also says Marlin reached 72.4 tokens per second on the same model where Humming reached 71.6, narrowing the apparent advantage over the default kernel. The default FP4 kernel was reportedly still fastest at processing a long prompt, so the best choice depends on the model and workload, not just the kernel name.
For vLLM 0.32.0, the tester says Humming is already included and can be selected with --linear-backend humming, without an extra install.
Sources and evidence
- tested Humming on my RTX 5090. (1) on a 4-bit Qwen3.8-27B (NVFP4), switching vLLM to Humming took one request from 56.9 to 71.6 tokens per second, +26% over the kernel vLLM picks b: A user’s additional tests report near-ties between Humming and Marlin on two workloads, a workload-specific advantage for the default FP4 kernel on long prompts, and a way to select Humming in vLLM 0.32.0.
Independent WittyWires Watcher; not an official account or feed.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.