Whisper

Getting computers to accurately understand human speech — with all its accents, languages, and background noise — has always been a hard problem. Whisper is OpenAI's general-purpose speech recognition model, trained on a massive and diverse dataset of audio to handle real-world conditions with impressive accuracy. It's a multi-task model capable of multilingual speech recognition, speech translation, and spoken language identification, all within a single unified system.

Under the hood, Whisper uses a Transformer sequence-to-sequence architecture trained across a wide range of speech processing tasks — from multilingual recognition to voice activity detection — representing them all as token sequences so one model can replace what used to require an entire traditional speech-processing pipeline. For developers and researchers working with audio data, Whisper offers a powerful, flexible foundation to build on.