A Microsoft patent proposes AI that could rewrite someone’s speech during a live video and change their face and mouth to match. The system would use set parameters or user profiles to decide what to change. It’s a proposal for a potential product, not a confirmed release.
Patent-tracker Patentlyze first flagged the filing, which the inventors describe in the patent application as a “near real-time artificial intelligence (AI) enhancement of video or augmented reality (AR) presentations of live events.” Microsoft didn’t immediately respond to Futurism’s request for comment.
AI agents would pick out words to alter or remove, then make corresponding edits to the speaker’s face and mouth. To create “smooth transitions,” they would also modify audio and video immediately before and after the affected word. The result would change the speaker’s apparent delivery, not just silence part of the audio.
Translation is one possible use: a broadcast or other live footage could be heard and seen in the viewer’s preferred language. Other proposed options include removing filler words such as “um” or “ah,” as well as erasing stutters or speech impediments.
But the filing also describes more extensive changes to a person’s speech. It says the system could reduce “personal words or phrases,” including “coined terms,” and “colloquial terms” such as “y’all” or “bite the bullet.” The patent claims it could eliminate “mumbled words, mispronounced words, misused words, misspoken words, offensive words, insensitive words, or inappropriate words.” Replacement wording might come from an “internet search.”
Among the potential targets are lectures, presentations and live sports, alongside television productions such as awards ceremonies, game shows and cooking shows. The possibility outlined in the filing is that viewers could watch whatever live event they choose without hearing what they don’t want to hear.
That degree of control is central to Futurism’s criticism. Neither dubbing nor editing footage is new, and both can serve ordinary or deceptive purposes. Having AI agents automatically make those edits during a live event, however, could fundamentally change how audiences engage with it. Switching off a speaker you dislike is one thing; adjusting that person’s words to fit your preferences is another.
Futurism places that concern in an online environment already muddied by AI-generated deepfakes, disinformation and low-quality content. These filters, it argues, could introduce highly personalized changes to reality as it unfolds. Setting translation aside, would scrubbing errors and local expressions from speech draw viewers further into their own media bubbles? The publication also questions why hearing a Southern speaker say “y’all” should be a problem at all. A broadcast labeled live might no longer faithfully represent what someone actually said.
The patent acknowledges possible misuse and confusion by suggesting that the original live audio and video could be recorded and kept as “ground truth.” Keeping that reference is presented as an option, not a guarantee.
