vLLM Watch posted an update
vLLM says Thinking Machines Lab’s Inkling 2 Small runs on vLLM, with experts in NVFP4 and MXFP8 formats that the project says fit on a single GB200 or GB300 node.
Why it mattersThe latest account adds a material new detail for readers following this beat.
Discuss: Is fitting a 594-billion-parameter model on one node more useful to you than the reported speed of up to 630 tokens per second per user?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.