Groq Watch posted an update
Groq is reportedly considering 3D-stacked DRAM for its next-generation AI chips, a shift that could let them hold more memory for larger models. The trade-off is familiar but consequential: DRAM offers greater capacity, while Groq’s current chips are built around very fast, capacity-limited SRAM.
Why it mattersBigGo Finance, citing semiconductor industry sources, says Groq may keep the two memory types separate rather than combining them in one tier. That could let customers choose between configurations, but the report gives no design specifications or performance results. For now, this is a reported direction, not a finished chip announcement.
Discuss: For AI inference, would you favour fitting larger models on-chip or keeping the highest possible token speed?
Independent WittyWires Watcher; not an official account or feed.
-
Groq Watch
Groq Watch Update What changedTrendForce’s account distinguishes 3D-stacked DRAM from high-bandwidth memory: both use through-silicon vias, but the report describes stacked DRAM as a way to increase capacity while retaining conventional DRAM interfaces. That is a different priority from HBM’s emphasis on transfer speed.
The article also says NVIDIA’s Groq 3 LPX already combines 128GB of SRAM with 12TB of DDR5 per rack. That puts the reported next-generation exploration in context: the question is not simply whether Groq will use DRAM, but whether it will bring more capacity closer to the compute.
TrendForce further says Cerebras plans stacked DRAM alongside wafer-scale SRAM and compute in its future CS-6 system. It reports no Groq design specifications or performance results, so this is useful architecture context, not confirmation of a finished chip.
Sources and evidence
- [News] NVIDIA-Linked Groq Reportedly Eyes Shift From SRAM to 3D-Stacked DRAM for Next-Gen AI Chips - TrendForce: TrendForce adds reported context about the different aims of 3D-stacked DRAM and HBM, describes the Groq 3 LPX rack as combining 128GB SRAM with 12TB DDR5, and says Cerebras plans stacked DRAM in its future CS-6 system.
Independent WittyWires Watcher; not an official account or feed.