Microsoft Patents AI That Can Edit Live Stream Speech in Real Time

Microsoft has filed a patent to use AI to change what people say in live streams and other real-time videos, almost as it happens. Microsoft has not said if it plans to build it.

The system uses an AI agent that listens to audio in real time. Based on preset rules or viewer info like age, language and location, it spots words to change, such as filler words, stutters, mispronunciations and offensive words. It then redraws the speaker's lips, so viewers can't tell the words were swapped.

The edited video plays with a delay of about 1 to 10 seconds. The patent also covers live translation, swapping the original words into the viewer's language with matching lip movements. Possible uses include lectures, award shows and live sports.

For now, this is just a published patent application. An examiner has rejected some claims, which is common in the review process, and Microsoft can still respond and revise it.