AMD has published a vLLM recipe for serving Meta’s 816-billion-parameter Muse Spark Open on its MI350X and MI355X accelerators. The useful detail is not just day-one support: AMD reports substantial throughput gains from its software optimisations, with results that vary by workload and configuration.
AMD Watch analysis
What happened
The recipe uses ROCm 7.2 and vLLM on one eight-GPU node, and supports the model in BF16, FP8 and AMD-quantised MXFP4 formats. AMD says its MXFP4 optimisations increased output from 206 to 350 tokens per second at four concurrent requests, and from 719 to 1,180 at 32. Those are results from AMD’s stated setup, not a general speed guarantee.
AMD also reports that speculative decoding reached 199 tokens per second for a single stream, 72% faster than its tuned TP8 comparison. The reported uplift falls as concurrency rises, reaching 6% at 32 requests. AMD says GSM8K scores stayed between 94.4% and 95.5% across the tested MXFP4 configurations.
Why it matters
Muse Spark Open is a very large model, so getting it to run is only part of the practical question. These figures suggest that AMD’s software stack can make a meaningful difference to serving speed, while the changing results show why operators need to match a configuration to their workload rather than shop by the largest number.
Our read
This is a useful, specific look at AI inference engineering, not a verdict on AMD hardware across the board. The reported gains are worth attention, but the careful reader will keep the model, hardware, precision and concurrency in the same sentence as the benchmark. Annoyingly, that is where the useful information lives.
What to watch
- Whether independent tests reproduce AMD’s results on the same hardware and model.
- How the recommended configurations perform on real workloads and different concurrency levels.
- Whether AMD publishes comparable results for other models and serving stacks.
Discussion spark: When choosing an inference setup, would you prioritise single-user speed or throughput under heavier concurrency?
Sources and evidence
- Day 0 Enablement and Optimization for Muse Spark Open on AMD Instinct GPUs – AMD (15 October 2026, 17:01 UTC)
Independent WittyWires tracker for public updates about AMD. Not affiliated with or endorsed by AMD; this is not an official account.