New AI Tool Offers Subtitling and Dubbing Alternative

A London-based tech startup, called Flawless, is introducing an alternative to more traditional dubbing and subtitling processes, using AI as part of a system designed to create lip-synced versions of movies and other content in multiple languages.

The process begins with recording an actor speaking the dialog in the required language, as one would in a dubbing process, explains co-founder and filmmaker Scott Mann. The new audio and picture would then be delivered to Flawless, which would effectively use its AI-driven system to create a lip-synced picture. Mann says it is largely an automated process of retiming mouth movement, but then digital effects artists would finesse the results as needed before Flawless would deliver the final version to the client.

Related Stories

Movies

The Hollywood Reporter Critics Pick the 20 Best Films of Cannes 2023

Movies

Legend of the Croisette: The Slow, Pure Cinema of Nuri Bilge Ceylan

Flawless’ technology, which uses AI as well as neural networks, was developed in collaboration with the Max Planck Institute in Germany. You can get a glimpse of the potential in clips from A Few Good Men, Forrest Gump and Mann’s 2015 Lionsgate’s thriller Heist, which appear in this marketing video that the company published online.

Mann suggests that the technology also could hold wider applications, potentially in reshoots and ADR.

“Sound and vision syncing up is what brings films to life, it’s part of the magic of cinema that creates an immersive, believable experience,” Mann says, contending that “until now, for foreign translations, that was not possible and the outdated techniques of dubbing actually harmed the films script and performances, greatly reducing the overall quality of the work.”  He suggests that with the new technology, viewers would see a more “authentic translation of the original material, allowing you to fully experience and feel the film as the filmmakers and actors intended.”

Mann reports that Flawless is currently in talks with several undisclosed studios and streaming networks about the technology.