beforesandafters.com
How the painterly style of The Wild Robot was realized. An excerpt from issue #25 of befores & afters magazine.For DreamWorks Animations The Wild Robot, based on the book by Peter Brown, writer/director Chris Sanders sought to bring a painterly aspect to the 3D animated film. The studio capitalized on stylized workflows it had developed previously on Puss in Boots: The Last Wish and The Bad Guys to take things even further for The Wild Robot, which follows the service robot Roz (Lupita Nyongo) who is shipwrecked on a wildlife-filled island, and who eventually becomes the adoptive mother of an orphaned goose, Brightbill (Kit Connor).In this excerpt, visual effects supervisor Jeff Budsberg gets into the weeds with befores & afters about exactly how the painterly aspects of the film were made, the tools developed at DreamWorks Animation, and, importantly, what this process added to the storytelling.b&a: Jeff, you really elevated things again with the stylization here, but its actually even a different approach to what Puss in Boots: The Last Wish and The Bad Guys did. What was your brief?Jeff Budsberg: I had come off of The Bad Guys as Head of Look and I was talking to Chris Sanders and Jeff Hermann and, right off the bat, Chris was interested in this space. Computer graphics has been in this pursuit of realism, which has been amazing and theres been so much innovation. But theres something that weve lost along the way, specifically in animation. If you go back to those 40s and 50s animations like Snow White, Bambi, Sleeping Beauty, theres something endearing about feeling the artists hand. I was just watching 101 Dalmatians with my kids this weekend and just feeling the stroke of the drawn line, the imperfections there. Theres something thats magical or just endearing about that. You feel the craft in there. And its not just that. Its also being deliberate about where there is information and where youre guiding the eye. Theres something about that that we really were interested in.issue #25 animation conversationsI think the other part is when I talk to directors and producers about stylized CG films is that really you dont just want to put Spider-Verse or The Bad Guys onto another film. It doesnt work. It doesnt make sense. You have to find the style that makes sense for that film, right? And thats one thing that Chris and I talked about a lot at the beginning of the movie is that we wanted to deposit Roz on this island, like a fish out of water. She doesnt belong there. So, on the surface, obviously she doesnt belong there, but we didnt want her to belong there aesthetically. Shes this precise, futuristic machine in a very loose, painterly deconstructed world, and immediately theres a juxtaposition there that is a conflict. And so you feel that she shouldnt belong there, but she really doesnt feel like she belongs there. And through the course of the film, she gets beat up and dirty and banged up and all that. So theres a progression of her wear, but we wanted there to be a very subtle progression of her aesthetic as well.And thats what was really exciting because it comes back to serving the story, in that she slowly starts to make an impact on the island with this relentless pursuit of kindness trying to help everyone. And, at the end of the day, the island is actually impacting her as well. So her aesthetic is changing sequence by sequence ever so slightly. And if we did it successfully, youre not paying attention to that. But then at the end of the movie, when Roz comes into contact with Vontra and the other robots, youre like, Holy crap, she now fits in the world and they do not. Now its jarring for them, because they dont belong there.Thats what was really exciting, because we were able to weave in this aesthetic that supported the storytelling in a really novel way. We evolve other things as well. Rozs locomotion changes from something that is more rigid or efficient, robotic-y, for lack of better nomenclature, into something thats more fluidic or animalistic, like S-curves with her arms. She does less peacocking; she uses less of her futuristic tools as she moves through the film. Shes doing all these crazy light shows and using all these things at the beginning of the movie. But then as she progresses, shes more restrained, a little bit more subtle in her mannerisms. So theres all these subtle cues that the audience might not pick up on. Its almost imperceptible, but the amalgamation of them, you feel her progression through the film. I think thats what was really rewarding aesthetically is you could use that, the style, to drive the story.b&a: What were the tools that DreamWorks already had in the bag to do this, but then where did you take things further? Doodle is a tool, for example, that enables you to do a lot of 2D animated elements as part of the 3D. Where do you take it here?Jeff Budsberg: We were using Doodle to some extent in visual effects. We had this idea, coming out of The Bad Guys and Puss Boots: The Last Wish, that you wouldnt ever make a realistic bush and then filter it to make it look painterly. It doesnt make sense, its not efficient and it doesnt give you a very pleasing result. So if you want a painterly bush, paint the bush, right? The takeaway there is, find the best place to solve the problem where every department along the way had to adjust their workflows to make the final image. So, on the film, modeling is not meticulously constructing a network of branches and leaves, they are drawing the bush in 3D with Doodle. It lets them think about shape language in strokes, splatters, and brushstrokes. The leaves dont need to connect, they could be splatters of flowers and color.Similarly for look dev and surfacing, think about how an artist would paint volumetric shading. What are the non-physical shaders that you need so that youre adding high-frequency texture in the key light, but on the shadow side, removing a lot of that superfluous information. Take for example bark on a tree; we would actually paint the lit side of the tree with a different texture than the shadow side of the tree. How do you build that into the renderer where you could swap textures and detail on the fly based on the light conditionsthats what we had to solve.Then theres feathers and fur. You want the richness and sophistication of real fur and feathers because you want to be able to see the fur moving in a sophisticated way. You want to be able to feel characters running their hands through fur, fluff their feathers, or have wind in there. You want to have these micro details that are amazing with fur and feathers. But we dont want to see every fiber of the hair. Thats not how one would paint it. So how do you reveal detail very surgically in specific areas? And how do you do that in a way that might not correspond to the geometry at all? Perhaps you want to add splatter or spongy texture in the key light of the fur, youre like, It doesnt even make sense. So your geometry might be fur, but were using brushstrokes to inherit the render properties that would drive the derivatives, the normals, the opacity. So, we use all sorts of different geometry to manipulate the light response driving non-physical shaders. Similar to a painters approach our shaders start with a really rough underlayer of loose detail and then youll add textural details on top of that as different accents, and those are only revealed through specific lights.b&a: I was talking to Chris Sanders about how actually doing it that way, its more stylized looking, but in some ways, its more believable.Jeff Budsberg: I think it comes back to what I mentioned before is that a handcrafted quality allows the audience a way to enter the film and fill it in their brain. It gives you a more immersive, imaginative experience compared to something where every leaf was detailed out, or every piece of grass. Its almost like your brain takes a step back. Youre like, Oh, this is so much visual information, I need to take a step back. But I think thats why we all go to the gallery to view paintings; you get to feel and experience a world through the glasses of the artist. You get to see the vision through their eyes and experience it with them. And I think it brings you something thats a little bit more of a visceral experience. And maybe thats what Chris is getting at when it feels more real because I think it feels just more inviting, a little bit more endearing. It feels handcrafted, it feels well-thought-out.b&a: How were you dressing the set, especially the island, on this film?Jeff Budsberg: We used a tool called Sprinkles, which is an art directed set dressing tool. I think one of the key developments for dressing the world is this integration between set dressing and modeling and being able to design bespoke plants on the fly and draw them. Were living in this interesting space between 2D and 3D. So, you could be in 3D dressing plants, but then in 2D, just drawing the plant, placing them, and youre living in this world where you bridge between illustrator and 3D artist. For Doodle, youre building these animation rigs on the fly as well. So all these plants could be articulated, they could all be interacting with the character and blow in the wind.b&a: I think thats what maybe people who are not familiar with the 3D process probably dont realize. Theyre like, Well, if its so 2D, why not just draw it?Jeff Budsberg: Exactly.b&a: But the characters brush past it, the camera moves.Jeff Budsberg: You step on the plants, you brush past them. This is one thing we talked about a lot is that we can give the audience something that you cannot do in 2D. You can do those dynamic camera moves. You can move through the space. You can do wind or deformation of the plants. You can do art directed depth of field, like rack focuses. There are things that you can do in the 3D world that you cannot do in the 2D world. And I think that allows you to move through the space in a way that is immersive and really inviting. But on the 2D side, theres something endearing about the handcrafted-ness. So, if we can live in this space between the two, I think that gives you something novel and really exciting. And I think thats what people grab onto like, It looks like a painting, but its moving. What is going on? This is crazy.b&a: When you mentioned a moving painting there, that is how I felt sometimes watching the film.Jeff Budsberg: Yes. The other key developments we made were adding brushed partial-transparency everywhere. So thats the other very typical thing of a CG render is that every edge is very hard and crisp. In a ray tracer, its very expensive to use transparency or even just onerous because we would want to do some processing in compositing, but a lot of compositing operations require AOVs like depth information or positional information, normals. But the problem with traditional renders is you only have one sample of the position at that pixel. But what if you have transparency? There are actually multiple objects at that pixel. You might rely on deep images, but deep images have significant problems because theyre theyre expensive to render and it makes your compositing very slow. You also lose pixel filtering and a lot of other features that you have in your traditional AOVs.So, we actually created an extension to the Cryptomatte data format where its a layered approach to your data channels. So every pixel in a Cryptomatte has an ID of what asset it is and the coverage information of that pixel. But then you store multiple layers, you have the sum of all of the assets at that pixel. Well, we decided why dont you do that for position? Why dont you do that for the normal? Why dont you do that for other data channels? Because what that allows us to do is use the really novel smart filtering operations from Bad Guys or Puss in Boots, in Wild Robot with transparent assets.We were able to use really sophisticated filtering operations, but with layered transparency, which would be normally very difficult to do. We werent really using a whole lot of transparent assets on Bad Guys and Puss in Boots. Having feather transparency and broken up edges really added to that believability that it felt like a brush had been applied and that texture was running over the page.b&a: What other things were you doing to make it feel more painterly?Jeff Budsberg: We had a scene sprites tool, where the lighters could decide that they wanted to perturb the image even more. Everyones reaching across disciplines in novel ways. The modelers are thinking about drawing plants. Surfacers are set dressing, could also draw the plants, and are thinking about lighting. Character effects, they usually do the groom, but the grooms influence the final aesthetics. Everyones thinking about the final image. The lighters, theyre authoring new assets with these sprites that are adding textural detail into the scene. But then they could use those in compositing to manipulate the image in a way thats still coherent spatially and temporally. And so they could add splatters of paint in the environment to help break up edges. We call that badger brushing, where you smear edges almost with a bristle brush.They could take those assets which are already pretty stylized, but they can push it even further and be like, Okay, well this shot needs some sort of painterly depth. Im going to layer in a couple of different textures of these scene sprites in the scene. They exist in 3D space so that the camera can move through the space and the characters could interact with the space and we could use those deep Cryptomatte-based filtering operations to push and smear the frame around like a painter would by painting wet-on-wet.b&a: Theres a lot of birds in the film, what did you have to do in terms of feather development or even crowds here?Jeff Budsberg: There was a lot of novel feather development in the rig. We actually developed new approaches for scapular feathers in the back. Theres a really fascinating way when birds fold their wings where this scapular, you can think of it almost like a cape where they fold their wings out and youre like, I dont even know where the wing went. It folds in a really novel way and so we wanted to build that into the rigs. And same thing with the pocket. So trying to put the wings in the pocket, which is really fascinating how they nuzzle them in and the wing disappears.In terms of other novel developments in the rigging for birds, theres this flap of skin called the propatagium. We were trying to simulate how that deforms and stretches as the birds are extending their wings. Those are some of just a handful of new developments for the bird rigs. And then for crowds, obviously its scaling that to thousands and thousands of birds. So, you have your fully fleshed out rig, then you have multiple simplified rigs along the way.But that wasnt necessarily the largest challenge. The largest challenge is, How do you simplify the geometry but then still make it feel painterly? You want to start to remove the same way you would in a painting, the birds that are further away, theyre almost look like blobs of paint, right? So youre trying to remove a lot of that high-frequency information. So, its the same thing where were using a lot of those tools I talked about, those scribbles and smears and processing the geometry in a way thats temporally stable and also respected to how far these things away from camera.That required a lot of investigation of, how do we achieve what we love about the Studio Ghibli films and Miyazakis world, but we want to live in this world of 2D and 3D? You wouldnt just use a 2D tool and just paint everything because that doesnt allow you the sophistication of a 3D world. So, Doodle allows us to take the 2D stroke data and project it in the 3D space, allowing you to spawn new simulations off of those. You can draw an explosion, which could be an emitter into a smoke simulation. And then that smoke simulation could affect brushstrokes through it, and then you could smear the brushstrokes in compositing. So you have this really interesting blend of 2D, 3D, simulation, compositing, and youre going back and forth almost like a painter would. Youre building up the image in a way that youre trying to use the best of 2D and 3D.If we need something like an ocean or a lake surface, we could use some sort of FFT or procedural way to deform the ocean or the water, but you dont want to see every micro ripple. So, we thought, we could process the geometry so you dont see all those high-frequency ripples, but then draw in some splashes. Its really interesting as an effects artist, to try to push the image in a way that does not need to be physically accurate, but it is physically believable. It is all the same motif of editing the image through the artists lens and trying to make it feel like it was handcrafted.Read the full coverage of The Wild Robot, and several other animated features, in issue #25 of befores & afters magazine.The post If you want a painterly bush, paint the bush appeared first on befores & afters.
