How Wt FX crafted the lead characters facial performance in Deadpool & Wolverine with Nuke and some machine learning techniques, and how they were also called upon to reanimate Wolverines corpse.When Deadpool was released in 2016, one of the ways the characters emotions were conveyed from behind the superhero mask was through visual effects augmentation of the costume itself, a task completed by Wt FX. That was also the case on Deadpool 2 (2018) and, most recently, Deadpool & Wolverine (2024). Not unexpectedly, the techniques to tackle that work have evolved over the three films.On the first film, we had a combined Maya to Nuke workflow, explains Wt FX visual effects supervisor Daniel Macarin. That involved our artists lighting and animating the mask in Maya and moving that through a standard process. Then just at the end of the first Deadpool, we started looking into a more 2D approach and keeping more of the practical that we could. We refined that in Deadpool 2 and advanced it to where we werent using any other package but Nuke.On Deadpool and Wolverine, continues Macarin, we refined that further, mixing both machine learning into the pipeline, as well as refining our relighting system and adjusting angles to make sure that we were able to rebuild and keep the resolution of the plate and the fabric and everything that we were touching no matter what direction that any character moves.Breaking down the facial animation workflowWt FX was solely responsible for this Deadpool mask work in the film. The process started with a relatively traditional matchmove of the head or anything the head and face needed to interact with. This was done in Nuke. Once we got that matchmove, the artist would work directly in Nuke to animate the mask, outlines Macarin, who notes that the facial setup in Nuke is connected to Wt FXs standard facial system of around 128 face shapes. Also, weve been making some tech refinements over the last few shows to make sure that the artist isnt having to clean up a dramatic amount. The only thing that theyre concentrating on is the performance.Plate.FinalWt FX has orchestrated its workflow to deal with any kind of plate that includes a facial performance from Reynolds or a stunt performer. Sometimes hell have the actual black leather on the mask, says Macarin. Other times he wont have any black leather and itll just be clear and it has to be replaced. Then theres the stuntie mask, which is more of a very thin fabric mesh that you can see through almost the entire time.In the platedepending on the kind of mask wornthe movement in the mask from mouth and other movement was something Wt FX directly used in its own animation, as Macarin discusses. In terms of the mouth movement, this is actually helpful to us because any movement from the mouth or the rest of the face guides us into the performance that we want to add. We want to link those up so that you understand that everything is connected.Its the same with body movement or position, adds Macarin. If the face is doing something that the body isnt doing, youre going to quickly read that something is off. You may not know exactly whats wrong, but youll see that something is wrong. The facial movement on a plate is only difficult for us when someone says, Wed like to change the line, wed like them to do something else. And then youre trying to readjust the jaw, stabilize everything, get rid of any lip movement or facial movement, bring it back to a neutral set and then animate on top of that.One of the important aspects for Wt FX in animating Deadpools facial performance is to not simply stretch the costume fabric, or make the animation cartoony. To ensure this did not happen, the visual effects studio endeavored to retain the plate as much as possible. The plates have so much detail in them, states Macarin. The on-set crew and the art department and the costume department go to such great lengths on these costumes. I think the audience really reacts to how well theyre designed, so much so that we would worry that going CG might potentially lose some of that detail. The more we keep, the better we feel about doing that.The machine learning sideArising from a suggestion by Wt FXs compositing supervisor on Deadpool 2, the visual effects studio investigated how machine learning could be used to aid in the facial animation of Deadpools mask for Deadpool & Wolverine. The idea was, relates Macarin, what if we took the audio and we process it in a way that it looks at all of the finalised keyframes of similar audio height levels and volume levels and tries to find what it thinks the facial performance should have been.The result provided an estimation of what Deadpools mask could be, given the inputted dialogue. The animator in Nuke could then use this as a starting point for the final nuanced animation. Interestingly, this machine learning audio approach proved to be extremely valuable in terms of environmental stimulus, as Macarin explains.Anytime someone shuts a car door or you hear an explosion in the background, the eye does a little tweak or a twitch. That wasnt something we wouldve normally hand- animated because its just not part of the dialogue. But when we had those machine learning paths, which had thousands of keys, you could scale down what it was doing and you got this really, really nice ambient motion to the face that gave it a much more natural feel and allowed the artist not to have to do thousands of little micro keys to try and get those really subtle movements. It helped give a little more life to the character than just straight facial performance.The tricky task of Wolverines maskDuring the film, Wolverine (Hugh Jackman) also dons his iconic yellow mask, and here Wt FX was called upon to also add facial expressions. The process had different challenges to Deadpools mask. In fact, initially, no augmentation on Wolverines mask was going to occur, since the mask featured stiff metal pieces and parts of Jackmans face were visible.Plate.Final.However, Macarin and the team at Wt FX felt that some movement in the mask would help communicate Jackmans trademark Wolverine performance. Having grown up with the comics and even cartoons like X-Men 97 and its earlier counterpart, Wolverine has certain facial expressions that I really felt that we could bring out. Also, when you look at Hugh when he plays Wolverine, he often has a very, very particular scrunch in his inner brow that is there most of the time, and you can really tell the emotion that hes feeling at the moment when he does that. When you didnt have that, we were concerned you were losing something. So, we did a test.Wt FX sent the test to production. The decided outcome was that the metal sections of Wolverines mask would not be moved, or only in very small ways. The yellow fabric between the metal parts, and the eyes, could be animated. Now, being that you are taking a plate within this mask and if you scrunch it down, theres nowhere to grab, advises Macarin. So, we had to rebuild the outside sections of the eye anytime his eye got smaller. Thats a challenge we didnt have with Deadpool.In terms of how far to take the eyes, we showed production tests where we went all the way to comic book eyes. They looked at it and they said, It looks really cool as a still frame, but if you watch it in motion, it comes across as a cartoon character. So, we slowly ramped it down and found a percentage of how much comic book we could put in that still brought the character to where we wanted it to be.Reanimating Wolverines corpseA slightly stranger animation task was also given to Wt FX for the films opening titles sequence, which features Deadpool digging up a variant of Wolverine from his grave. Here the studio needed to digitally augmented Wolverines corpse, something Deadpool talks to and even uses to fight a number of adversaries with.Plate.Final.The original idea was to have a zombified body, but it was going to bring up too many questions, like, would he heal? details Macarin. So for on set, they built an adamantium skeleton with almost no skin anywhere at all, and thought that was the easiest solution for the audience to not ask too many questions.During editorial, it was deemed funnier to give more life to Wolverines corpse. Wt FXs art department developed concepts that considered adding more remaining skin to the skeleton and versions with a mummified eyeball. They sold the idea to us with a very specific moment, recalls Macarin. They said, For Hughs credit, were going to rip the skin oI of his face and then you see Hugh Jackman underneath. The shots all slow-mo so theres all this stuff going as the skins tearing.The decision to add more skin and pieces to the corpse meant that Wt FX had to matchmove the skeleton prop that had been used on set, and then replace it. Macarin was confident the extra CG effects elements added the desired extra humor to the sequence. After the first shot they saw the CG additions in, which was Ryan playing with Hughs chin and talking to him, they said, We love it. This is going to work. Everyone was really happy.Final.The post Behind the mask: how intricate facial animation was added onto Ryan Reynolds Deadpool appeared first on befores & afters.