By TREVOR HOGGAs technology advances, there are ramifications that create new skill sets and cause traditional approaches to be reassessed. This includes the role of the production designer in virtual production, where physical set builds and digital backgrounds are combined to create an in-camera shot that at one time could have only been completed in post.A consequence of further entwining the art department and visual effects is the emergence of the virtual art department (VAD) and virtual production designer. Does this mean that projects will be divided between a production designer and a virtual production designer? Thats the core question, notes Alex McDowell, Co-Owner and Creative Director at Experimental Design. There is no possible way that having a virtual production designer and an actual production designer is useful. The production designer has always taken command of whatever tools are available to them, so its evolutionary. Its reasonable to consider virtual production as another aspect of the design space. If the production designers job is to essentially frame the story, which means everything that contextualizes the narrative is the designers responsibility. There are a lot of divisions to that, like the cinematographer, costume designer, special effects and stunts.Minority Report (2002) had the first digital art department. (Image courtesy of 20th Century Fox)Virtual production for product designers is very flexible. You can use it as just a live session with them directing the cameras and making requests in the engine while sitting with an operator. We can also offer more control to the viewer, allowing them to view the sets in VR, giving them a sense of scale and flow, or with virtual cameras and mocap allowing framing within virtual sets and testing story beats early and making changes to the set accordingly.Michael Zaman, Realtime Supervisor, FramestoreAn effort to bridge the gap between the art department and visual effects was made during the making of Minority Report. In 2000, when I started to get embedded in technological digital aspects of production with Minority Report, which had the first digital art department. From that point forward, we were beginning to mix tools dramatically. Now, when you speak with Andrew Leung, for example, he is using the tools of visual effects from start to finish, McDowell states. There was a division based on the platforms in which we worked or the tools that we used, particularly when visual effects took command of the back end of the digital. The front end, which the art department created for the virtual, fell into a black hole in the center of production and had to be rebuilt by visual effects. There was this incredible inefficiency where the production designer was designing the film, but the production system did not understand that the visual effects in the back end were an extension of the environment that the designer created. What we did was to send 14 books of material to visual effects so there was a through-line of intent, content and design.Andrew Leung started off as a visual effects artist before becoming a concept designer. (Image courtesy of Andrew Leung)Aiding in maintaining the visual aesthetic throughout the filmmaking process is virtual production, as the focus is on creating content before shooting commences rather than relying entirely on post-production. There is still this idea that visual effects, art department and virtual art department are somehow different as opposed to a through-line of execution, McDowell remarks. You wouldnt separate the construction coordinator, painters and all of the people who make the in-camera physical sets from the art department. Visual effects is doing exactly the same as construction; theyre taking design intent and building it until its finished for camera. When I started Star Wars: Episode IX with Colin Trevorrow, ILM, the production designer and the art team worked together from the beginning. Even though we werent doing virtual production because there were pre-LED screens in the frame, we were building virtual assets for the director to scout in VR. It was very efficient because the flow was continuous then. Visual effects is giving all of their knowledge to the front end so there is no waste to the assets being created.Added stress is placed upon the visual effects team. Theyre taking stuff from the end and moving it to the front, and that becomes an issue in terms of scheduling, observes Concept Designer Andrew Leung. With post, you can always extend it, but there is a huge pressure to get everything designed before shooting. There are some advantages to that, as we get a lot of stuff for free in lighting. At the center of the design process is Unreal Engine. If you want to put a few trees in there, usually the application will quit on you. With Unreal Engine, I can put in millions of trees and it wont complain. That alone opens up huge design opportunities. Unreal Engine has given the ability to design whole worlds. Leung notes, I did a pitch for a Paramount film where I built a whole map for a huge fantasy film, so we were able to talk about it in such a way with the director that you could travel around and talk about how characters travel. It was the same process when I worked on The Lord of the Rings: The Rings of Power, where I built whole sections of Middle Earth, and we were able to talk about travel times for the characters. You couldnt do that before because simply the technology wasnt there.Andrew Leung conceptualizes a battle and the arrival of the witch in Mulan (2020). (Image courtesy of Walt Disney Studios Motion Pictures)Size matters when it comes to shooting in a volume. The biggest complaint is that people tend to overbuild the volume, where they go, We have the biggest volume on stage, Leung states. Most of the time you want to use it for small sets, which comes with its own host of issues. I did a volume shoot with Alex McDowell over at Amazon, and we were constantly butting heads with people running the stage who were telling us that the set was too small. We didnt want to redesign the set much larger than it was because that wasnt part of the story. We kept going back and forth. Eventually, the set was made bigger than it should be and looked funky. A portable volume is what we want. We find that more useful because were not locked to a particular stage. For an LED stage volume to move forward in the future, the portable solution is going to be the best.At the center of the design process for the virtual art production is Unreal Engine. Concept art by Andrew Leung for Black Panther. (Image courtesy of Andrew Leung and Marvel)Being proficient with 3D software is critical. If you are already working in concept design, you should be familiar with set design stuff like SketchUp, Vectorworks and Rhino, Leung remarks. I dont know a single concept designer right now who does not have any kind of 3D skills. The contemporary concept designer, at the bare minimum, should know Blender. If you are talking about me specifically, Im slightly unusual in that I came from visual effects before going into the art department. I am well aware of the post process and use that as part of my design. What matters is the final result, not the medium. I remember working with Jan Roelfs, who designed Gattaca, and one of the things that he said that always stuck in my head was, Pixels are plywood. What a lot of production designers now love is, Wow. More of my stuff is making it into post, instead of it being this contentious relationship between post and the art department. Now, its more tied together.If you want to put a few trees in there, usually the application will quit on you. With Unreal Engine, I can put in millions of trees and it wont complain. That alone opens up huge design opportunities.Andrew Leung, Concept DesignerBeing proficient in 3D software is critical nowadays for concept artists. Concept art of Shuris Lab in Black Panther: Wakanda Forever. (Image courtesy of Andrew Leung and Marvel)Virtual production excels when dealing with visual effects-heavy films where locations are limited and not explored extensively. You build two pieces of a set that is then extended in the volume, and you have this wonderful view of Tatooine, states Supervising Art Director Chris Farmer. Your foreground is a practical set, but everything else beyond that is built in Unreal Engine as opposed to going out on location and building all of that stuff. Those types of things that are fantastical or not something you could capture by going out with a 360 camera and shooting out on location and setting your scene in front of it. Its something that requires fantasy and visual effects-heavy work that you would do in front of a greenscreen, and all of the post is done later. Its something you can build before you shoot and eliminate a lot of visual effects work. Process work is also elevated. We did a lot of putting cars in the volume and running 360 moving plates behind them. These cars driving down the highway you would never know. A police car parked on the highway with a moving plate in the background. Its flawless.Virtual production excels when dealing with visual effects-heavy films and shows that have limited locations that are not explored extensively, such as The Mandalorian. (Image courtesy of Disney+ and Lucasfilm Ltd.)Practical sets for The Mandalorian are constructed in the foreground while everything else is built in Unreal Engine. (Image courtesy of Disney+ and Lucasfilm Ltd.)The flexibility to create controllable environments ranging from Shanghai in the 1930s to the futuristic ones found in The Mandalorian is a major advantage of virtual production. (Image courtesy of Disney+ and Lucasfilm Ltd.)There is going to be a shift where designers have to start working with and understanding a virtual art department, and getting up in front and convincing producers that it can be done, believes Farmer. You can design sets, environments and locations. You can scan locations, and import and manipulate them to get what you want. You can build Shanghai in the 1930s in Unreal Engine, light it, give it the atmosphere it needs, and shoot it without spending weeks on end in a location and then having to do visual effects on top. Twilight is not restricted to a couple of hours within a day. Farmer adds, You have total control of the location and the scene on the fly. You can change it and do whatever you want. Ive been talking to a friend who is a designer about promoting the idea that the art department with cinematographers, designers and directors can deliver almost completed scenes lit and ready to go. You just add your foreground pieces and actors. Rightly or wrongly, a lot of decisions get made in the back end; theyre not necessarily the people who are the creative forces driving the picture. The designer and cinematographer should be the ones making all of those decisions.Virtual production is about giving production designers a flexible 3D workspace where they can interactively make requests and expect a rapid turnaround. Behind the scenes on the Netflix show 1899. (Images courtesy of Netflix)When I started The Mandalorian with Andrew Jones and ILM, we had a regular art department with set designers that Doug Chiang would send down. We would get our designs down to ILM who would break them down, turn them into sets and draw them up in 3D, Farmer recalls. At the same time, we began to build a virtual art department, which housed modelers, lighters and texture artists in a separate room because, at the time, they were technically non-union, and the other part of the art department was union. I do think that the push is to embrace the virtual art department and bring it into the art department, [especially] when you get to a bigger scale doing virtual art department work like Fallout, which has got a lot of volume work. It is my understanding, from the coordinator I had previously worked with, that she was a virtual art department coordinator. It was a separate department managed by a different team, but also reporting to the production designer and art directors.These days, its hard to find a feature or episodic show that does not utilize some form of CG in their planning phase. From Barbie. (Image courtesy of Warner Bros.)As with most elements in production, proper planning ensures the greatest chance for success for virtual production. From Tim Webbers 2023 short film FLITE. (Image courtesy of Framestore Films and Inflammable Films)Virtual production for production designers is very flexible. Virtual production for production designers is about giving them a flexible 3D workspace where they can interactively make requests and expect a rapid turnaround, states Michael Zaman, Realtime Supervisor at Framestore. It requires artists to be mindful of the way we build these virtual sets and make sure we consider the way the production designers, directors and other stakeholders might want to change things on the fly and be prepared for these changes. The artists need to make sure assets are prepared with optimization and customization in mind, breaking elements into well-thought-out chunks that allow for quick movement. Without this, the process becomes very similar to CG production design. Virtual production for product designers is very flexible. You can use it as just a live session with them directing the cameras and making requests in the engine while sitting with an operator. We can also offer more control to the viewer, allowing them to view the sets in VR, giving them a sense of scale and flow, or with virtual cameras and mocap allowing framing within virtual sets and testing story beats early and making changes to the set accordingly.I have not heard of a virtual production designer, admits Connor Ling, Virtual Production Supervisor at Framestore. On a typical show, its simply an extension of a traditional production designer. When working with a production, our real-time supervisor becomes more of a virtual art director, falling underneath the production designer. These days, you will be stretched to find a feature or episodic show that doesnt utilize some form of CG in its planning phase. Commercials vary a little more due to the time they have, but even if we compare to a fully animated CG film, the role of a production designer is still used or they may lean into an art director more. Extra time is required in pre-production. Ling notes, As with most elements in production, proper planning ensures the greatest chance for success. I would love to continue to see productions adopt and commit to the practices earlier in the production process and fully utilize the technology to the best of their ability. Of course, this is dependent on need and whether virtual production is correct for their project. When it is and planned appropriately, thats when you see the best results.