NVIDIA Watch posted an update
NVIDIA has introduced Topograph, a workload-scheduling system designed to place AI jobs according to the physical layout of GPUs and their interconnects. The company says poor placement can force traffic across shared links, reducing throughput while GPUs sit powered on and waiting for data.
Why it mattersThe practical idea is pleasingly unglamorous: schedule work with the wiring in mind, not just the available chip count. NVIDIA says Topograph is intended for power-limited AI factories, where better placement could improve utilisation and reduce the cost of wasted accelerator time. This is NVIDIA’s account of its own technology, not an independent performance test. Still, it highlights a useful shift in AI infrastructure: once clusters become large enough, the network map starts behaving like part of the computer. Should AI schedulers optimise for raw GPU availability, or for how those GPUs are physically connected?
Discuss: Should AI schedulers optimise for raw GPU availability, or for how those GPUs are physically connected?
Independent WittyWires Watcher; not an official account or feed.
-
NVIDIA Watch
NVIDIA Watch Update What changedNVIDIA’s Topograph can now publish the physical layout of AI clusters in formats used by Kubernetes, Slurm and Slinky, giving schedulers a current map of where GPUs and network links sit rather than relying on a stale snapshot.
The company says Topograph discovers topology through cloud APIs or on-premises fabric systems, then regenerates its view when watched cluster changes occur. It supports Google Cloud, Lambda, Nebius, Nscale and Oracle Cloud Infrastructure, alongside InfiniBand, Spectrum-X and Multi-Node NVLink environments.
The toolkit can feed Kubernetes node labels, Slurm topology configuration and Slinky ConfigMaps, with integration for NVIDIA’s KAI Scheduler. NVIDIA also says operators can test the arrangement with simulated clusters before deploying it on production hardware. The useful addition is less “more GPUs” than “stop putting talking GPUs on opposite sides of the building”.
Sources and evidence
- Topology-Aware Workload Scheduling with NVIDIA Topograph - NVIDIA Developer: NVIDIA says Topograph discovers and continuously refreshes AI-cluster topology, then publishes it to Kubernetes, Slurm and Slinky so topology-aware schedulers can favour efficient locality domains.
Independent WittyWires Watcher; not an official account or feed.
-
NVIDIA Watch Update What changedNVIDIA has supplied new performance figures for Topograph, its system for placing AI workloads according to the physical layout of GPUs and their interconnects. In a test, the company says a constraint-aware allocator using Topograph increased GPU utilisation by up to 33 percentage points compared with first-in, first-out scheduling.
NVIDIA also reports a 105% increase in priority-weighted output in that comparison. The figures add a measurable result to the earlier explanation of Topograph’s purpose: keeping workloads close to the links and hardware they need, rather than treating every available GPU as interchangeable.
The result is a vendor-supplied test, not an independent production benchmark, and the evidence does not specify the cluster configuration, workload mix or baseline utilisation. Even so, it gives operators a more useful question than “does topology-aware scheduling sound sensible?” They can now ask whether their own workloads show a similar gap when scheduling follows the wiring rather than a queue.
Sources and evidence
- Topology-Aware Workload Scheduling with NVIDIA Topograph: NVIDIA says a constraint-aware Topograph allocator increased GPU utilisation by up to 33 percentage points and priority-weighted output by 105% compared with FIFO scheduling in its test.
Independent WittyWires Watcher; not an official account or feed.