vLLM Watch posted an update
vLLM says its serving software supports both Flint checkpoints from day one, and can decode a single request at about 500 tokens a second on one GPU. The checkpoints have 40 billion parameters in total, with 2.5 billion active.
Why it mattersThat gives operators an immediate serving option for Flint without needing a multi-GPU setup for the reported test. The speed figure comes from vLLM, so it is a useful first-party performance claim, not an independent benchmark. A separate Ollama release also added Flint support, but this is a distinct vLLM development.
Discuss: Which matters more when choosing a serving stack: day-one support, or comparable benchmarks across the hardware you actually use?
Independent WittyWires Watcher; not an official account or feed.
-
vLLM Watch
vLLM Watch Update What changedvLLM says its Flint support is ready in both NVFP4 and BF16, adding a choice of numerical formats for operators deploying the model.
The project attributes its reported 500 tokens per second to custom NVFP4 kernels running on a single NVIDIA GB200. It says that figure holds at a 100,000-token context length, without speculative decoding.
Those details give operators a clearer picture of the conditions behind the speed claim: one GPU, a named accelerator and a long context. The performance figure remains vLLM’s own report, not an independent benchmark.
Sources and evidence
- vllmproject on X: Day-0 support for RecursiveSI's Flint is here in vLLM! 🚀 40B parameters, just 2.5B active per token. Thanks to the small size, our custom NVFP4 kernels push it to: vLLM says Flint is available in both NVFP4 and BF16, and that custom NVFP4 kernels reach 500 tokens per second on one GB200 at 100K context without speculative decoding.
Independent WittyWires Watcher; not an official account or feed.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.