Xiaomi MiMo-V3 to Use New HySparse 2 Architecture, Cutting Prefill Compute by 5x

Luo Fuli, head of Xiaomi's MiMo large model, announced that MiMo-V3 will soon adopt a new architecture, with the core component HySparse 2 officially released. At 1 million token length, the new architecture reduces prefill compute by 5.02x, shrinks KV cache by 4.5x, while achieving higher MRCRv2 and RULER-v2 scores and lower AgentPPL and LongPPL.

Luo noted that agentic reasoning is a completely different workload: in each interaction round, a short action may return long text observations that need prefill, while context keeps growing. This puts prefill cost, KV cache size, and retrieval accuracy on the critical path simultaneously. HySparse 2 uses KV Bridging and KV Reuse to make all cross-decoder KV caches come from the self-decoder, so prefill can stop once the self-decoder finishes.

Additionally, Xiaomi released and open-sourced the Xiaomi MiMo-V2.6 series the day before, including Pro and Flash native omni-modal models. Xiaomi calls this a key step in exploring the RSI path, expanding model intelligence through scaled-up reinforcement learning compute.