Groq, the AI chip startup acquired by NVIDIA, is reportedly planning to redesign its chip memory architecture. South Korean outlet ZDNet reported on October 20 that Groq is evaluating the use of 3D packaging in its next-generation chips, replacing SRAM—the core of its existing design—with DRAM to significantly increase the memory capacity of each chip.
Groq’s Next-Generation Chip Design Takes a New Direction: DRAM May Replace SRAM
ZDNet, citing two anonymous semiconductor industry sources, reported that Groq is reviewing the use of 3D packaging in its next-generation chips, replacing the SRAM in its current design with DRAM. One industry source said Groq plans to use 3D packaging in its new chips, stacking DRAM instead of continuing to use SRAM.
3D packaging vertically stacks multiple semiconductor dies so they operate like a single chip. Reports indicate that compared with traditional 2D planar designs, this approach can reduce footprint and may also improve power efficiency.
Groq was founded in 2016 by Jonathan Ross, one of the designers of Google’s TPU, and was acquired by Nvidia last year. Previous reports that its “Groq 3 LPX” had entered mass production suggest that Groq’s product line continues to advance within Nvidia.
What Is 3DS? How It Differs from HBM in Capacity
Reports indicate that Groq’s approach is reportedly 3DS (3D Stacked Memory), which uses through-silicon vias (TSVs) to vertically connect DRAM dies, forming a high-capacity package. HBM also uses TSVs, but has a different design goal. HBM focuses on data transfer rates, while 3DS retains the existing DRAM interface and focuses on expanding capacity.
Groq is not the only company advancing memory-stacking technology. As Chain News previously reported, SK hynix has begun commercializing 3D-stacked DRAM-on-Logic. The technology stacks DRAM directly on top of logic chips, shortening physical distances and adding data channels to overcome the bandwidth and power-consumption limitations of current PoP packaging. It is primarily aimed at AI on mobile devices.
The difference lies in the architecture. At present, all that is known about Groq’s 3DS is that it stacks DRAM to expand capacity; it has not been specified whether the DRAM is stacked on top of the compute die.
Why DRAM? A Capacity Trade-Off Suited to AI Inference
ZDNet notes that Groq’s previous designs centered on SRAM, which is fast but offers limited capacity. DRAM is relatively slower, but better suited to providing large capacities. The report interprets this move as an attempt by Groq to overcome the capacity limitations of its existing design. Another industry insider says the market as a whole is moving toward larger chips, so Groq has chosen DRAM, which offers an advantage in capacity.
From the perspective of AI inference, this trade-off makes sense. As LLM parameters and context lengths continue to grow, memory capacity requirements are also rising, and relying solely on SRAM can become limiting.
However, SRAM’s advantages are its speed and low latency. It remains unclear whether these characteristics can be maintained after switching to DRAM. Further details from Groq or Nvidia are needed to assess whether this design shift is real and what its practical impact will be.
This article is reprinted with permission from: (Chain News)
Original title: (Nvidia Moves into Next-Generation Memory Chips! Groq Reportedly Adopts 3D-Stacked DRAM to Meet AI Inference Demand)
Original author: Crumax
The article “Nvidia Targets AI Inference Demand! Groq Reportedly Adopts 3D-Stacked DRAM to Develop Next-Generation Memory Chips” was first published on “Crypto City”.