Startup Launches Prometheus AI Server to Challenge Nvidia's GPU+HBM Combo

Startup Majestic Labs has unveiled the Prometheus AI server, which uses AIUs (Ignite AI Processing Units) with LPDDR6 memory instead of Nvidia's GPU and HBM combo. Each server can have up to 12 AIUs and 8TB to 128TB of LPDDR6 memory, combining Arm cores with RISC-V vector and tensor engines. A standard 40U rack can hold four Prometheus servers, with a total power draw of 120kW, using liquid cooling. The company claims one Majestic rack has as much memory as 25 Nvidia NVL72 Vera Rubin racks, while using less power. The server is expected to launch next year, is OCP-compliant, and works with PyTorch, vLLM, and OpenAI Triton.