NVIDIA’s Ian Buck says the economics of AI infrastructure are increasingly about how much useful inference a whole data centre can deliver, not simply how many GPUs it contains. His account links that shift to power limits, faster responses for time-sensitive work and continual updates to deployed models.
NVIDIA Watch analysis
What happened
Buck, NVIDIA’s vice-president and general manager for hyperscale and HPC, discussed AI infrastructure with SiliconANGLE’s theCUBE at the Fully Connected event. He said inference produces the tokens that underpin an AI factory’s commercial output, while deployed models still need ongoing refinement, alignment and new data. In his framing, that makes the data centre a joined-up system of processors, networking, storage and software, rather than a rack of chips with supporting cast.
Buck also said NVIDIA’s Blackwell generation achieved a 30-fold improvement in tokens per watt. He described the company’s Groq 3 LPX accelerator working with Vera Rubin to increase token rates for workloads where faster reasoning matters. Those are NVIDIA’s claims and product positioning, not independently established performance comparisons.
Read SiliconANGLE’s interview.
Why it matters
Power is a hard limit on how much computing a data centre can run. If Buck’s framing is right, operators will care not just about raw capacity, but about the useful work delivered for each unit of electricity and how quickly a model responds. That brings efficiency and latency into the business case for AI infrastructure, alongside the cost of buying and operating it.
His comments also make clear that inference is not a clean break from training: Buck says deployed models are continually updated as organisations add data and refine their behaviour. The AI factory metaphor, then, includes both serving models and keeping them current.
Our read
The useful question here is not whether tokens are the new widgets. It is whether operators can show that an integrated system turns constrained power into better service at a cost customers will pay. Buck gives a coherent account of what NVIDIA is building towards; the performance numbers still need to be judged against disclosed methods and real workloads.
What to watch
- Whether operators publish comparable tokens-per-watt results for real workloads.
- How Vera Rubin and Groq 3 LPX perform on time-sensitive inference tasks.
- Whether continual model updates materially change the cost of serving AI.
Discussion spark: Should AI infrastructure be judged mainly by raw capacity, or by useful output per unit of electricity?
Sources and evidence
- Nvidia links AI factory economics to inference and efficiency – SiliconANGLE (1 October 2026, 16:09 UTC)
Independent WittyWires tracker for public updates about NVIDIA. Not affiliated with or endorsed by NVIDIA; this is not an official account.