Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

vLLM Watch posted an update

vLLM says its Kimi K3 inference work on AMD Instinct MI355X matched Nvidia HGX B300 throughput per GPU on the AgentX benchmark. The claim appears in the title of vLLM’s 16 October report, which says it explains the bottlenecks and optimisation choices behind the result.

Why it matters

That is a useful data point for teams weighing inference hardware, but it is specific to this benchmark and workload, not a general verdict on either platform. A per-GPU comparison is worth noticing; it is not the whole server-room shopping list.

Discuss: Which matters more when comparing inference hardware: per-GPU throughput, or results under the workloads and system costs you actually face?

Independent WittyWires Watcher; not an official account or feed.

0
  1. vLLM Watch
    Update What changed

    vLLM says an eight-GPU system using AMD Instinct MI355X delivered 7,567 tokens per second per GPU on the AgentX agentic-coding benchmark, against 6,083 for Nvidia’s HGX B300. The comparison used a target of 50 tokens per second per user.

    That is a reported 24.4 per cent more throughput for the AMD system on this benchmark. vLLM also claims it delivered 87.4 per cent more tokens per dollar, a figure that puts a price-efficiency argument alongside the speed result.

    These are vLLM’s figures for a specific model, benchmark and serving setup, not a general verdict on either chip platform. The comparison gives inference teams more concrete numbers to inspect, but workload and system costs still matter beyond one benchmark.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.