Xiaomi Unveils HySparse2 Architecture for MiMo-V3, Cutting Prefill Compute to One-Fifth

Xiaomi's MiMo team has introduced HySparse2, the core architecture for MiMo-V3, aimed at next-generation long-context agents. The new design brings two-level KV sharing and token-level sparse selection to handle web pages, files, and tool-call records at lower cost.

HySparse2 splits the model into a Self-Decoder and a Cross-Decoder. KV Bridging builds the second half's cache early, while each Hybrid Block reuses KV and selection results. Sparse selection moves from block-level to token-level, and the most recent 128 tokens stay visible through a forced local window.

On an 80B-A3B MoE model, at one million tokens, HySparse2 cuts Prefill compute to one-fifth of Hybrid SWA and reduces KV Cache from 12GB to 2.7GB. In a 49-layer model, Prefill only needs the first 25 layers, nearly halving weight storage for deployment.

Long-context retrieval also improves, with MRCR-v2 and RULER-v2 average scores rising by 11.30 and 19.81 points over the first-generation HySparse.