Nvidia's GB300 smashes MoE training record with 3x performance leap

Nvidia says its Blackwell GB300 system set a new world record for MoE (Mixture of Experts) model training, hitting 1648 TFLOPs per GPU. MoE splits a big model into smaller expert networks, only using some at a time to save power and cost.

Using 256 GPUs to train the DeepSeek-v3 671B model, the GB300 NVL72 hit 1648 TFLOPs per GPU, matching performance with fewer chips. Nvidia simulated over 250,000 setups to find 38 big improvements, running 1.4 million hours of GPU tests.

That's a 50% jump from November 2025's 1088 TFLOPs, and nearly 3 times faster than last-gen GB200's 606 TFLOPs. The result shows how hardware and software tweaks work together.