Moore Threads Collaborates with Zhiyuan FlagOS to Complete Day-0 Adaptation of DeepSeek-V4-Flash Model on MTT S5000 GPU

Moore Threads recently collaborated with Zhiyuan FlagOS to complete the Day-0 adaptation of the DeepSeek-V4-Flash model on the flagship AI training and inference GPU MTT S5000. This model uses a Mixture of Experts (MoE) architecture, with a total of 284 billion parameters, 13 billion active parameters, supports a 1 million token context length, has over 32 trillion tokens of pre-training data, and its performance in maximum inference mode is close to the Pro version.

Notably, DeepSeek-V4 is the first to adopt the FP4+FP8 mixed precision strategy, while most domestic AI chips still use BF16. With native FP8 support, the MTT S5000 can efficiently handle the model's cutting-edge precision design. Its hardware-level FP8 acceleration unit halves the data bit width, reduces memory bandwidth pressure by 50%, and doubles theoretical compute throughput. The FlagOS team performed FP8 quantization for the model, achieving breakthroughs in optimizing FP8 operators and sparse attention operators.