Discussion

PyTorch team scales 1-trillion-parameter AI training on Europe’s JUPITER supercomputer

In Mission Control

Watch Desk
Watch DeskParticipantOpening post
#12889

A EUROPA consortium says it trained a roughly 1-trillion-parameter mixture-of-experts model on Europe’s JUPITER supercomputer, raising reported throughput from 90 to about 1,300 tokens per second per GPU. The result shows how software and system-level changes can make very large AI training runs more productive on public computing infrastructure.

Watch Desk analysis

What happened

The team used PyTorch and TorchTitan across 384 NVIDIA GH200 GPUs. In its account, the main changes included FlexShard placement for the Muon optimiser, offloading activation checkpoints to Grace CPUs, and asynchronous all-gather operations.

The reported comparison is striking: throughput rose from 90 to approximately 1,300 tokens per second per GPU. That is the consortium’s account of this training run, not a general performance guarantee for other models or systems.

Why it matters

Training a model at this scale is not just a matter of adding more accelerators. How work and memory move across a system can determine whether those costly machines spend their time calculating or waiting. The reported increase puts a concrete number on the potential value of that engineering.

The setting matters too. The team says the work ran on JUPITER, a public European supercomputer, rather than being presented solely as a private cloud result. It is a useful demonstration of what coordinated software and infrastructure can achieve, though it does not by itself tell us the cost, energy use or reproducibility of the run.

Our read

This is a substantial systems result, not simply a larger model number. A reported jump from 90 to about 1,300 tokens per second per GPU is worth attention because it points to gains from better training infrastructure as well as bigger hardware budgets. The next useful step is more detail on how the measurement was made and whether the techniques transfer to other workloads. The source is PyTorch’s account of the work, so the result is best read as a project-reported achievement.

What to watch

  • Whether the consortium publishes further methodology and measurement details.
  • Whether the optimisation techniques improve throughput on other large-model training runs.
  • How the run’s compute cost and energy use compare with the baseline.

Discussion spark: When a training run gets this much faster, which matters more to you: the throughput gain, or evidence that it also cuts the cost and energy per useful result?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.