GenStorAIGE's AI90 puts SSD into GPU memory, cutting first token delay by 50 times

GenStorAIGE showed its AI90 hardware-software AI inference acceleration solution at WAIC 2026. It puts high-performance SSDs into the GPU memory system, offloading KV Cache to solid-state drives to create a HBM+DRAM+SSD three-level memory setup, aiming to solve memory bottlenecks in large model inference.

The solution cuts first token delay from seconds to sub-seconds, speeding it up by 50 times. Throughput increases 5.1 times, and memory usage drops 39%. With smart P2P multi-card tech, AI90 boosts an 8-card NVIDIA GeForce RTX 5090 cluster by 5.8 times and supports 128K+ context.

To handle high write loads from KV Cache offloading, GenStorAIGE also launched the PT200Z AI SSD. It uses pSLC flash with a PCIe Gen5 interface, reaching up to 100 DWPD, sequential read of 14.8GB/s, random read of 3100K IOPS, read latency of 54μs, and write latency of 10μs.