Micron Develops Near-GPU NAND to Break VRAM Limits for AI

According to DigiTimes, Micron, a US memory chip maker, is developing high-endurance NAND flash modules to be placed closer to GPUs, shortening data access paths. This solution, called near-GPU NAND, aims to balance storage density and durability for workloads that don't need ultra-low latency. The NAND can sit inside the GPU package or on the PCB, working like HBM or GDDR7, letting the GPU directly access hundreds of GB of storage. It has lower density than traditional TLC or QLC NAND but focuses on faster I/O and bandwidth, acting as a high-speed buffer between VRAM and regular storage. This could help large AI models run without being limited by VRAM capacity, potentially impacting the AI industry.