AMD Sets New MLPerf Inference Record, Nvidia's Vera Rubin Makes Debut

In the MLPerf Inference v6.1 benchmark, AMD and Nvidia submitted their latest results. The benchmark, maintained by MLCommons, measures throughput and latency for AI inference tasks. Version 6.1 focuses on inference throughput, scaling efficiency, and performance across different GPU setups.

AMD used 512 Instinct MI355X GPUs and achieved 5.75 million tokens per second in the DeepSeek R1 test, its largest submission to date. Offline results reached 2,901,950 tokens/s, while server results hit 2,405,310 tokens/s.

Nvidia submitted GB300 and GB200 with 288 GPU configurations in the same test. GB300 scored 2,705,130 tokens/s offline and 2,028,030 tokens/s on server. Nvidia's Vera Rubin VR200 also debuted in the benchmark, running 95% faster than GB300 in a 72 GPU setup.