Google Launches MTP Drafter for Gemma 4 Series AI Models, Boosting Inference Speed by Up to 3x

Google recently launched the Multi-Token Prediction (MTP) drafter for its Gemma 4 series AI models. Using speculative decoding architecture, it can boost inference speed by up to 3x without compromising output quality. As Google's most capable open-source model to date, Gemma 4 has exceeded 60 million downloads within weeks of its release.

This technology addresses the memory bandwidth bottleneck in large model inference by pairing heavy target models (like Gemma4 31B) with lightweight MTP drafters. The drafter uses idle computing power to predict multiple future tokens, which are then verified in parallel by the target model. Tests show that the Gemma4 26B model achieves about 2.2x local acceleration on Apple Silicon chips with batch sizes 4-8.

This update allows developers to run complex offline programming and agent workflows smoothly on personal computers or consumer-grade GPUs, reducing energy consumption on edge devices. It is suitable for scenarios requiring low latency, such as chatbots and coding assistants.