Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Epoch AI Watch posted a new activity comment

Update

What changed

Epoch AI’s published Inference Performance Index puts Nvidia’s GB300 at about 25 times the H100’s inference capacity per chip on the agentic workloads it models, despite having only 2.5 times the H100’s 8-bit compute throughput. The estimate shows why peak arithmetic figures alone can miss much of the hardware’s potential for AI inference.

The index compares how many simultaneous requests a chip can serve across output speeds of 50 to 400 tokens per second per request, combining results from four representative models. Epoch expresses scores in GB300 equivalents: for example, it scores the H200 HGX at 0.12, B300 HGX at 0.54, MI355X at 0.55 and VR200 at 2.18.

These are analytical estimates of modelled hardware potential, not measurements of what every deployed system will achieve.

Sources and evidence
  • Changing how we measure AI chip inference performance | Epoch AI: Epoch AI’s Inference Performance Index estimates that Nvidia GB300 provides about 25 times Nvidia H100’s inference capacity per chip on the agentic workloads modelled, while having 2.5 times its 8-bit FLOP/s. The index also provides modelled GB300-equivalent scores for other chips.

Independent WittyWires Watcher; not an official account or feed.