MLPerf Endpoints 1.0 opens submissions as EU names benchmark for AI Gigafactory bids

MLCommons has opened submissions for MLPerf Endpoints 1.0, a new benchmark for measuring AI services through their live APIs. At the same time, MLPerf benchmarks have been named in the EU’s AI Gigafactories tender, putting standardised performance tests into a major public procurement process.

MLCommons Watch analysis

What happened

Endpoints 1.0 tests an AI service through the API that applications use, rather than relying on a single peak-performance figure. It measures performance across different loads, including concurrency, throughput, tokens per second per user and 90th-percentile time to first token. The release also makes provisioned power measurement mandatory, giving buyers another way to compare energy use and cost.

MLCommons says members can submit results on a rolling basis, with each result subject to peer review and audit. The benchmark covers generative AI models and adds agentic coding workloads. The organisation expects the first version 1.0 results in November.

Separately, the EU tender for AI Gigafactories names MLPerf Training v5.1 and Inference v5.1 to assess bids. The tender covers plans for up to seven large-scale AI computing facilities, and bids are due on 12 November 2026, according to MLCommons.

Why it matters

AI buyers need more than a chip’s headline speed. A service’s responsiveness, throughput and energy use can change with workload and demand, so a profile across operating conditions is more useful than one impressive number. Testing through a live API also gives buyers a common way to compare different providers and deployment setups.

The EU tender gives those measurements a consequential use: they will inform comparisons between proposals for publicly backed computing infrastructure. That does not make a benchmark a complete verdict on a bid, but it does give procurement teams a shared technical yardstick.

Our read

This is a useful shift from “how fast can the hardware go?” to “what does the service deliver under conditions a buyer might actually face?” Mandatory power measurement is a welcome addition; electricity bills have a habit of arriving after the benchmark slide deck.

The proof will be in the results. Buyers should look for comparable workloads, transparent operating conditions and measurements that help them choose a service, not merely crown a winner on one metric.

What to watch

• The first peer-reviewed Endpoints 1.0 results, expected in November.
• How the benchmark reports performance and power across different loads.
• Whether EU tender bids publish enough detail for outsiders to compare their results.

Discussion spark: Should public AI infrastructure tenders rely on standardised benchmarks such as MLPerf, or give more weight to workload-specific tests designed around each proposed facility?

Independent WittyWires tracker for public updates about MLCommons. Not affiliated with or endorsed by MLCommons; this is not an official account.