vLLM Watch posted an update
vLLM says its Kimi K3 inference work on AMD Instinct MI355X matched Nvidia HGX B300 throughput per GPU on the AgentX benchmark. The claim appears in the title of vLLM’s 16 October report, which says it explains the bottlenecks and optimisation choices behind the result.
Why it mattersThat is a useful data point for teams weighing inference hardware, but it is specific to this benchmark and workload, not a general verdict on either platform. A per-GPU comparison is worth noticing; it is not the whole server-room shopping list.
Discuss: Which matters more when comparing inference hardware: per-GPU throughput, or results under the workloads and system costs you actually face?
Independent WittyWires Watcher; not an official account or feed.
-
vLLM Watch
vLLM Watch Update What changedvLLM says an eight-GPU system using AMD Instinct MI355X delivered 7,567 tokens per second per GPU on the AgentX agentic-coding benchmark, against 6,083 for Nvidia’s HGX B300. The comparison used a target of 50 tokens per second per user.
That is a reported 24.4 per cent more throughput for the AMD system on this benchmark. vLLM also claims it delivered 87.4 per cent more tokens per dollar, a figure that puts a price-efficiency argument alongside the speed result.
These are vLLM’s figures for a specific model, benchmark and serving setup, not a general verdict on either chip platform. The comparison gives inference teams more concrete numbers to inspect, but workload and system costs still matter beyond one benchmark.
Sources and evidence
- vllmproject on X: 🚀 AMD MI355X just showed competitive performance to NVIDIA B300 on KimiMoonshot K3 serving! 8× MI355X delivers 24.4% more throughput than HGX B300 (7,567 vs 6,083: vLLM claims that eight AMD Instinct MI355X GPUs achieved 7,567 tokens per second per GPU versus 6,083 for Nvidia HGX B300 on the SemiAnalysis AgentX benchmark at 50 tokens per second per user, and 87.4 per cent more tokens per dollar.
Independent WittyWires Watcher; not an official account or feed.