Huawei open-sources 505B-parameter openPangu-2.0-Pro model

Huawei has officially open-sourced its 505B-parameter openPangu-2.0-Pro model, including weights, basic inference code, and a technical report. The model is a large Mixture-of-Experts (MoE) language model trained on Ascend NPUs, with about 18B active parameters per token, supporting a 512k context length and trained on 34T tokens. Post-training includes SFT, multi-task reinforcement learning, and online distillation. Architecturally, it features a DSA+SWA hybrid attention, a 4-branch mHC residual design, a 3-head MTP speculative module, and uses the Muon optimizer for faster convergence. Earlier, the 92B openPangu-2.0-Flash was released on June 30. Huawei executive Richard Yu noted that most of their compute capacity supports other domestic companies, but they focus on improving latency and throughput.