Next-gen Groq LPU chip may adopt 3D stacked DRAM

Groq's current LPU inference chip uses a large SRAM cache to deliver fast decoding output. But SRAM holds less data, and shrinking the process brings limited density gains. That puts real limits on large AI models and future chip upgrades.

According to Korean media ZDNET Korea, the next-generation Groq LPU AI inference chip is expected to use 3D stacked DRAM based on TSV (through-silicon via) technology, breaking the capacity bottleneck by stacking memory vertically.

Cerebras, which also took the big SRAM route, has already said its future CS-6 chip will vertically integrate DRAM in 3D. Both moves suggest this tech path is looking for a new answer to the memory demands of large AI models.