Nvidia's new TwoTower model speeds up text generation by 2.42x

Nvidia has launched Nemotron-Labs-TwoTower, a discrete diffusion language model that tackles the bottleneck of token generation speed. With 60 billion total parameters, it uses a TwoTower architecture: a 30B autoregressive context tower and a 30B diffusion denoiser tower, each activating 3B parameters with 128 routed experts.

The key innovation splits context representation and denoising into two separate neural networks. The context tower stays frozen to maintain autoregressive context, while the denoiser tower learns to denoise noisy blocks. They work together through layer-by-layer cross-attention.

According to Nvidia's tests, the TwoTower model retains 98.7% of quality but boosts generation throughput by 2.42x. It performs similarly to pure autoregressive models on benchmarks like MMLU and ARC-Challenge, while being much faster. The model is open-sourced on Huggingface.