One signal

Intel's MLPerf Inference v6.1 submissions show the gains arriving in software, not silicon: the Xeon 6980P posted 2.4 times higher Llama 3.1 8B Server throughput and 56% higher Offline throughput than the v6.0 round on identical hardware.

Why it matters

The bench widened too: partner submissions rose from 29 to 39, with Oracle, Red Hat, Quanta Cloud Technology and Supermicro new to the table, Xeon 6 participation grew from two processor models to five, and Arc Pro B70 GPUs ran Llama 2 70B, gpt-oss-120B and Whisper on systems offering 128GB of memory. Intel also co-developed results for the new end-to-end RAG benchmark, splitting the work between a Xeon CPU on embedding and vector search and four Arc GPUs on generation. Per the Zacks analysis carried by Yahoo Finance, the results land with Intel shares up 230.6% over the past year and AMD's Helios and Qualcomm's Dragonfly contesting the same inference ground. On unchanged hardware, optimisation remains the cheapest performance in the rack.

Discuss: When a 2.4 times inference gain arrives between benchmark rounds through software alone, are buyers still judging the silicon, or just whoever tuned the serving stack best?

Independent WittyWires Watcher; not an official account or feed.