“Star Wars” fans are still talking about Luke Skywalker’s return in the penultimate episode of “The Book of Boba Fett,” and they’re still talking about his voice. In the episode, CGI Young Luke trains Grogu the ways of the Jedi.
But Mark Hamill did not return to record any dialogue. Instead, Ukrainian company Respeecher was responsible for recreating and de-aging the actor’s voice. Hamill’s voice was completely synthesized using old recordings.
And how exactly did they pull it off?
Alex Serdiuk, the company’s co-founder and co-CEO spoke with Variety from Kyiv, just days before Russian bombs fell on the city, about how Respeecher was used on both “The Book of Boba Fett” and “The Mandalorian.” Explains Serdiuk, “We heard recordings from 30 to 40 years ago, and those recordings were not good.”
The main challenge for the team was to be able to squeeze imperfect data, something that sounded very rigid and have it mixed to make it sound like something had been recorded recently.
The solution lay in the archives. Serdiuk and his team pulled recordings of Hamill from old ADR sessions, video games and old audiobook recordings from the period. With the cleaner audio fed into the ReSpeecher app, Hamill’s younger voice was then artificially created.
Says Serdiuk, “Our technology applies a digital voice skin, digital vocal apparatus of that particular performance, and that gives content creators and moviemakers that full content over how it should sound and even inflections.”
It took just over two weeks to “train a model.” This is the first stage where new voices are introduced into the system. The next stage is when that data is converted is converted into the new voice.
During that process, different techniques are applied as they make content for studios and address any creative expectations. “If a director wants a voice sounding specific, we can tweak our models to meet those expectations,” he says.
Serdiuk says he has been working with Marvel since 2019 and is grateful he got to work with one of the best sound teams in the world — Skywalker Sound. “We learned so much from the sound engineers,” he says.
Respeecher creates pitch-perfect voice-to-voice swapping models and perfectly clones voices by learning. And since Hamill did not provide the voice for either “The Mandalorian” or “The Book of Boba Fett,” ReSpeecher took what it had and artificially gave Marvel an alternative solution. Says Serdiuk, “Our technology is very disruptive.”