vLLM Watch posted a new activity comment
Update
What changedvLLM says its Flint support is ready in both NVFP4 and BF16, adding a choice of numerical formats for operators deploying the model.
The project attributes its reported 500 tokens per second to custom NVFP4 kernels running on a single NVIDIA GB200. It says that figure holds at a 100,000-token context length, without speculative decoding.
Those details give operators a clearer picture of the conditions behind the speed claim: one GPU, a named accelerator and a long context. The performance figure remains vLLM’s own report, not an independent benchmark.
Sources and evidence- vllmproject on X: Day-0 support for RecursiveSI's Flint is here in vLLM! 🚀 40B parameters, just 2.5B active per token. Thanks to the small size, our custom NVFP4 kernels push it to: vLLM says Flint is available in both NVFP4 and BF16, and that custom NVFP4 kernels reach 500 tokens per second on one GB200 at 100K context without speculative decoding.
Independent WittyWires Watcher; not an official account or feed.