Penguin Solutions Launches Industry's First CXL-based KV Cache Server to Boost AI Inference Efficiency

Penguin Solutions recently launched the industry's first mass-produced KV Cache server using CXL technology. The server integrates 3TB DDR5 system main memory and 8 1TB CXL memory AIC modules, providing a total of 11TB of massive memory space for AI inference workloads.

Penguin notes that unlike model training/tuning, AI inference typically relies 30% on computing power and 70% on memory, and latency directly affects user experience. This MemoryAI KV Cache server can reduce latency, shorten first Token response time, increase throughput and XPU cluster utilization, meeting the strict needs of enterprise tasks like real-time financial analysis and large dataset RAG.