NVIDIA is reportedly putting Groq's specialised inference technology into a production rack headed for neocloud provider Nebius later this year. The significant bit is that NVIDIA has turned the technology into a rack-scale product intended to sit alongside its Vera CPUs and Rubin GPUs, giving cloud customers a new inference option within the NVIDIA ecosystem.
NVIDIA Watch analysis
What happened
The Groq 3 LPX rack reportedly contains 256 liquid-cooled Groq chips. LPX is aimed at low-latency inference, while NVIDIA's GPUs continue to handle training and large-context processing. The account comes from Yahoo Finance relaying CNBC and statements attributed to NVIDIA senior director Dion Harris, rather than a directly supplied NVIDIA announcement.
NVIDIA says the rack can deliver 3,400 tokens per second, citing a benchmark from Artificial Analysis. That is a company-reported reference, not a comparable independent result in the supplied evidence: the model, workload, latency definition, batching and rival configuration are not given.
Why it matters
For cloud buyers and AI infrastructure teams, this points to a more modular choice at the inference layer without demanding a wholesale escape from NVIDIA's software stack. The practical test will be whether Nebius makes the system available, at what price and with what real-world latency and throughput.
It also shows how quickly NVIDIA can commercialise acquired or licensed technology. But one named customer is not a market, and a specialised inference chip is not a replacement for a general-purpose training platform.
Our read
This is a meaningful early deployment signal, not proof of broad adoption or independently verified benchmark performance. Infrastructure teams should ask for Nebius availability, workload-specific performance, pricing and CUDA compatibility before changing hardware plans.
What to watch
- Nebius confirmation
Whether Nebius confirms the deployment and publishes an availability date. - Independent performance data
Whether comparable tests reproduce the claimed 3,400-token-per-second result. - Customer breadth
Whether other cloud providers or customers commit to Groq 3 LPX racks. - Operational compatibility
Whether CUDA compatibility holds in real deployments without material software or operational changes.
Discussion spark: Does integrating specialised inference hardware into NVIDIA's existing stack broaden practical accelerator choice, or mainly make NVIDIA's platform harder to leave?
Sources and evidence
- NVIDIA (NVDA)’s $20 Billion Groq Bet Goes Live This Year With New AI Racks (6 September 2026, 19:07 UTC)
Independent WittyWires tracker for public updates about NVIDIA. Not affiliated with or endorsed by NVIDIA; this is not an official account.