'The BFG' Visual Effects Whiz Joe Letteri Makes the Case for Motion-Capture Performances

THE BFG -Still 7-Sophie (Ruby Barnhill) and the Big Friendly Giant (Mark Rylance)-H 2016
Courtesy of Storyteller Distributuion Co.

Is an actor who delivers a motion-capture performance deserving of serious awards consideration?

The question was first raised when Andy Serkis played Gollum in The Lord of the Rings trilogy and again when he took on the part of Caesar in the last two Planet of the Apes movies. But with the arrival of The BFG, which opened Friday in theaters, the discussion has a new example that further demonstrates how an actor's performance can be combined with visual effects to create a compelling character.

In the case of Steven Spielberg's new movie, the 24-foot titular ‘Big Friendly Giant’ is the creation of newly minted Academy Award winner Mark Rylance (Bridge of Spies) and Weta Digital and its four-time Oscar-winning senior visual effects supervisor Joe Letteri

The Disney release is based on the Roald Dahl classic about a young girl (Ruby Barnhill) and the giant who introduces her to Giant Country. The film blends live action with performance-capture-based giants, filmed on both practical and virtual sets.

The giant is the latest amazing creature from Letteri and Weta, whose previous Oscar-winning work included Gollum, Serkis’ Kong in Peter Jackson’s King Kong and the whole cast of blue-skinned Na’vi in James Cameron’s Avatar.

Letteri also has supervised work on CG performances as far-ranging as the human-like simians of the Planet of the Apes reboot and the stylized cartoon-strip characters in The Adventures of Tintin. He also handled the delicate task of creating a digital Paul Walker in order to complete Furious 7 following the accidental death of the actor.

“Each character comes with its own challenges and you apply what you’ve learned to each new project so we’re constantly improving the quality level,” Letteri told The Hollywood Reporter. “Our recent work on the BFG required us to deliver a 24-foot giant whose performance was so close to what Mark Rylance did on set that you’re almost asking the audience to see through the digital character and emotionally connect with Mark.

“Mark used extraordinary subtlety in his performance of the BFG and it was up to us to understand how his acting manifested itself visually so we could make our giant convey the same thing,” he added. “One point that was really apparent to us when we were working on Furious 7 is that it’s vital to separate the actor and ‘the actor in character’ in your mind. They are inevitably similar but not the same. If an actor is off set and out of character, their expressions will be different than the ones they are giving to the camera in the role. In this way, there really is no pure BFG, there is only ‘Mark Rylance-as-the-BFG.’ If we’ve done our job, the audience will just get caught up in the character and the story.”

In filming The BFG, Spielberg used a ‘Simulcam,’ an idea originally conceived by Cameron and first used on Avatar. It's the process of combining real-world actors and sets with actors and sets that are computer-generated. “With Simulcam, we can pre-record a performance and then play it back through the camera monitor so that the camera operators could actually see the virtual performance unfolding in real time as they’re photographing the live-action scene,” Letteri explained. “By combining the two, they’re able to make decisions and frame and actually even cue actions based on what’s happening in the virtual world.”

Using this technique — which put these images in the hands of Spielberg as he used a handheld virtual camera device — the production filmed actors in performance-capture suits acting on the same set with the film’s human characters. That was especially important to the director because it allowed Barnhill and Rylance to interact with one another.

Based on their performances, the VFX team then created a believable computer-generated character. “What we’ve found is that it’s not just one thing, it’s really how all of the elements come together because your eye is so attuned to the human face and human expressions that you quickly notice when something is not right,” said Letteri. “The detail in the model and the complexity of the facial rig are just as important as the sophistication of the skin properties are all critical. And then you really need skilled artists because it’s what they’ve learned from studying the face that allows them to produce work that isn’t just successful from an artistic point of view, but a human point of view.”

In the cases of The Lord of the Rings, King Kong and Avatar, the painstaking work earned the VFX teams on those films Academy Awards, but the actors behind the computer-generated characters have yet to earn a nomination, despite campaign efforts, particularly in the case of Serkis for his work as Gollum in LOTR and Caesar in POTA, as well as Zoe Saldana in Avatar.

Could the first nomination for an actor in a role that involved performance capture be on the not-so-distant horizon? “I think most people would agree that we’ve seen worthy performances, but the issue is how to recognize them,” Letteri answered. “It’s inherently a partnership and those are difficult to award, but I think it will happen. Ultimately, audiences enjoy these characters and they have become part of the modern moviegoing experience, so finding a way to recognize the actors and digital artists that are doing the work only makes sense.”