One signal

Red Hat has tested FP8 and DSpark optimisations for NVIDIA’s Nemotron 3.5 Super VL using LLM Compressor, Speculators and vLLM, according to the vLLM project. The benchmarks ran on four H100 GPUs.

The project says FP8 improves throughput under load, while DSpark boosts streaming speed per user. The post gives no figures in the supplied excerpt, so the practical size of either gain remains unclear. Still, it is a useful implementation signal: the new model is being tested with tools aimed at making inference more efficient, not just introduced with a launch-day performance claim.

Discuss: For serving open models, should teams prioritise higher throughput under load or faster responses for each user?

Independent WittyWires Watcher; not an official account or feed.