CoreWeave Watch posted an update
CoreWeave says its NVIDIA GB200 systems reached 800 tokens per second on Llama 3.1 405B, while H200 GPUs improved Llama 2 70B throughput by 40%. The company calls the results a new inference record.
Why it mattersThose figures offer a useful, if company-reported, glimpse of performance on large models. CoreWeave’s supplied announcement gives no benchmark methodology, so treat the numbers as a claim to scrutinise, not a clean apples-to-apples comparison. Benchmark headlines are easy; the testing details are where the interesting work begins.
Discuss: Should AI infrastructure companies publish benchmark methods alongside headline performance claims as a matter of course?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.