The visual effects supervisor sheds light on the process behind developing the series fantastical realm.By Jay StobieThe Two Trees of Valinor in a shot from Season One (Credit: Amazon).Based on the work of J.R.R. Tolkien, Prime Videos The Lord of the Rings: The Rings of Power (2022-Present) stands out as a sweeping epic taking place during Middle-earths Second Age, a period which predates the timeframe depicted in The Lord of the Rings (2001-2003) and The Hobbit (2012-2014) film trilogies by thousands of years. In order to bring a cinematic scale to the streaming series, The Rings of Power team enlisted Industrial Light & Magics own Jason Smith to be the production visual effects supervisor.With a formidable rsum that includes work on everything from Star Wars: Revenge of the Sith (2005) and Transformers (2007) to The Revenant (2015) and Bumblebee (2018), Smith brought his accumulated expertise to bear as he guided ILM and the series other visual effects studios on their journey through seasons one and two. Smith graciously took some time to speak to ILM.com in order to discuss his work on The Rings of Power and highlight ILMs extensive contributions to crafting showrunners J.D. Payne and Patrick McKays vision for Middle-earth.An Epic EndeavorAs the senior visual effects supervisor on the project, my job is to work with the showrunners to help them put their story visually on the screen, Smith tells ILM.com. This monumental responsibility encompasses hiring vendor studios and planning out the work, being on set and instructing the shoot during principal photography, and shepherding the shots through post-production. So it kind of includes everything. Its a real privilege because, on this project, I was one of the first handful of people who joined. We got to take on some challenges in Middle-earth that hadnt been done yet, like Nmenor and the living Khazad-dm.With a background in blockbuster films, Smith has witnessed how studios approaches to episodic and theatrical productions have grown more alike in recent years. The bar has been raised by so many great shows over the last 10 or 15 years that the expectation is that [series] quality is film quality, Smith observes. Nevertheless, there are notable distinctions involved in making a series that Smith also treasures, such as the approval process and the collaborative spirit of the showrunners, in this case J. D. Payne and Patrick McKay. Theres quite a lot of freedom to throw out ideas, to suggest things, and to help to guide the visual storytelling with those guys. Our work with the directors is more about prep and being on set, truly. So thats a key difference there. It leads to a little bit more autonomy in post, which I really appreciate and love.Such a rapport is essential, especially given the volume of visual effects shots associated with The Rings of Power. Nothing can prepare you for the number of shots on one of these shows, Smith shares. Because lets say youve got 6,000 shots, and at the peak, maybe you have half of those in play. If you just imagine what its like when 3,000 shots have one-minute reviews crossing your desk, if you do the math, it catches up with you very, very quickly. It turns the whole thing into this ballet, and it gives me a lot of respect for the production teams at the vendors and here [at Amazon]. Its astoundingly good, they keep us all organized in spite of ourselves. Smith estimates that, in the first season, ILM worked on around 1,000 effects shots under the supervision of Nigel Sumner. ILM was our lead house on season one and did a lot of work, along with Wt [FX], Method [Studios], and others.Season Two concept art by Saby Menyhei (Credit: ILM & Amazon).ILMs InnovationsA capacity for collaboration proved to be a vital strength demonstrated by Smith and ILM, particularly when it came time for Mount Doom to unleash a volcanic eruption and transform the Southlands into the infamous region of Mordor. ILM played a huge role in the volcano. We went into that with complete ownership of it, Smith recalls. The volume of shots that came in for the sequence necessitated that the work involve three vendors. We had Weta doing some really big pyroclastic cloud shots that cut back to back with ILM shots, and ILM and Weta both cutting back-to-back with RSP [Rising Sun Pictures] for some pyroclastic clouds and raining lava bombs.The challenge of coordinating with other studios resulted in a clever solution whose ingenuity was defined by its simplicity. We treated all three companies like they were the same company, no secrets, sharing everything fearlessly, Smith states. For the vendors, sharing their works-in-progress amongst one another provided a common frame of reference for the elements they were bringing to life, whether that be the darkness of a cloud or the intensity of an ember. Everybody pulled it off amazingly well, Smith beams. Most people who see [those scenes] would never actually detect the different styles in play, as those shots cut back-to-back. Its a testament to those teams being willing to work together.Early in season one, the scenes involving Galadriel (Morfydd Clark) and Halbrand (Charlie Vickers) adrift on the Sundering Seas required yet another imaginative remedy from ILM. When we read the scripts for season one, an amazing amount of it took place on the ocean. So I immediately wanted to go to ILM with that work, Smith details. They created a setup we called it a machine so people outside of visual effects could sink their teeth into it it was a very robust procedural ocean that let them tune the [waters] parameters, like the wind and height of the waves. [ILM visual effects supervisor] Nigel Sumner was leading the charge on that. Smith was impressed by the Ocean Machine, as it allowed the team to turn out photoreal plates once the scenes parameters were set.However, Smith adds that of course, theres artistic work involved in every single one of those [plates]. In our show, every time the ocean is there, its telling a story, so we had adjustments to make across the board. At the extreme end of the spectrum , there was a gigantic storm one with an epic Middle-earth scale but all of that stuff came out of that same engine with the same artistry behind it. Smith teases that ILMs season one advances also paid dividends for water-related shots in season two. I will say that, without knowing exactly whats coming except for having read Tolkiens books [laughs] I would anticipate that we should be getting a lot more use out of the ocean machine.A Season One shot depicts Galadriel and Halbrand aboard wreckage on the Sundering Seas (Credit: Amazon).Season Two SensationsAfter praising ILM for its marvelous work across both seasons, the latter of which was supervised by Daniele Bigi, Smith shifts his focus to his favorite contributions from ILM that have shown up in the early episodes of season two. For instance, the shows sophomore effort affords us an opportunity to meet the Elven elder, Crdan (Ben Daniels), who resides on an idyllic shoreline. Theres a matte shot, an environment shot, pulling back from Crdans boathouse at the Grey Havens, and the work is just so photographic-looking, Smith elaborates. In an interesting twist, the tranquil locale was actually built upon a set standing in the parking lot next to an industrial park. The water is flawless, the fjords are so beautiful, the lighting is wonderful, and the composition is great. That is one [shot], when people see it, we already have people asking, Wait, where did you build that? They were wondering where that was [in the real world], so I think thats a testament to the artistry at ILM.Smiths gratitude for ILMs craft extends to one of season twos most complex sequences. The premiere begins with a flashback to Saurons apparent death, when his rival Adar (Sam Hazeldine) orchestrates a coup against him. Initially reduced to a viscous goo that oozes into Middle-earths cavernous depths, Saurons enduring form subsists on passing rodents and insects until it is able to free itself from its tomb. The squirming coalescence gradually makes its way toward civilization, consuming a traveler and allowing Sauron to assume the human form known as Halbrand. ILM had the guts to take on one of the weirdest things in the whole show, and I mean that with the most excitement that I can communicate, Smith says in regard to the eerie endeavor. It is such an incredible challenge of simulation, and creature rigging, and complex animation, and rendering, and basically every single discipline. [ILM] really hit the ball out of the park with that.Despite these fantastical qualities, Smith sees the importance of grounding such creatures with the properties of real-world references. We found a species of worm that formed colonies and that will contract almost like a muscle themselves, and we combined that with reference of horsehair worms, which are really unsettling in their own way. The final result is really beautiful to watch, and gut-wrenching in a way. It looks like animated SpaghettiOs. Were always looking for references like that, because those are the little handholds into reality that take these creatures to the next level. Storytelling is another key factor in ensuring that the visual effects showcasing Saurons resurrection resonate with audiences. Its so intertwined with the story, its exactly the story that needs to be told about that character. It interacts with an animal, too, which I think is also beautiful work. Thats one of the things Im excited for people to see.Concept art by Dave Freeman (Credit: ILM & Amazon).Final frame (Credit: Amazon).The Wonders of WorldbuildingSmiths passion for Middle-earth and its wealth of creatures can be traced back to his childhood. When I was growing up, the world of Tolkien was a huge deal to me. I loved it, Smith explains, believing Peter Jacksons movies to be some of the best ever made. They really captured my heart. And, being a creature kid at the same time, its a match made in heaven. Citing a popular Tolkien quote I wisely started with a map, and made the story fit that originated in a letter penned by the writer, Smith emphasizes the need to start with the realms foundation, whether that be its geographical details or the intricacies of a fictional language. When were doing fantasy, especially when its a little more imaginative, we have to have that anchor.From hypothesizing how Ents emote to his own attachment to the hill-troll, Damrod (voiced by Benjamin Walker and Jason Smith), Smith reflects back to the moment he learned hed get the chance to deal with more creatures in season two. Patrick McKay said to me that, this season, were going to battle a lot more creatures. Things are getting darker, and we have a story-based need to really bring some of these other creatures into play. A reverence for the natural world was also key for McKay and Payne, as their desire to remain enveloped in the story meant that any magic displayed in the series could not have too much, as Smith puts it, razzmatazz. Smith recounts that the magic of the first season involved beings who manipulated flames, kicked up winds, and threw rocks about. Its almost like the beings who are working with the magic have access to another chapter of the laws of nature, so what looks like magic is their interaction with more advanced, but still physical and grounded, laws, Smith postulates.Its all caught up into these elemental ties, Smith continues. Even on set, when were filming scenes with magic, Daniel Weyman, who plays the Stranger, and I would stand there and pick each others minds a little bit. [Id tell] him, I think that to increase the threat, Im going to have rocks picked up in the wind, so that when youre thrown against the wall, some rocks are hitting all around you. It makes it look like it really hurts. Weyman reciprocated the brainstorming sessions, and Smith himself remains grateful that hes able to bring his ILM experience to the table. I feel really lucky that I got into ILM render support when I did, and that Ive been able to work with the people that I have at ILM. To be mentored by people like [ILM visual effects supervisor] Scott Farrar, [ILM executive creative director] John Knoll and [ILM consulting creative director] Dennis Muren is a real privilege, and I take a lot of the wisdom those guys have said over the years into all of these different interactions and challenges.The Balrog from Season Two (Credit: Amazon).Travels and TriumphsCollaborating with the showrunners and cast members is not the only one of Smiths duties as the shows senior visual effects supervisor that audiences may be unaware of. I shoot all the aerials for [The Rings of Power], Smith reveals. Weve got some drone stuff that happens without me, but so far, Ill say, Ive shot all the helicopter work for the show. Which is great, because then when were designing shots, I have the shots in my head. Traveling around New Zealand and the United Kingdom, Smith found real-world sights frozen rivers, tundras, green valleys, cliffs, waterfalls, and more that fit with the series vision and could be inserted into its story.If you look at the big CG [computer-graphics] establishing shots, almost without fail, they are all based on those aerials, Smith declares. ILM handled such a shot in the season two premiere, Elven Kings Under the Sky, in which High King Gil-galad (Benjamin Walker), Galadriel, and Crdan don their rings of power under the cliffside tree. We see a final wide [shot] of that [scene]. That is just a plate we shot not knowing what exactly would be placed there [laughs], and I think ILM did beautiful work of making it look like it was all on purpose the whole time, placing the court of Lindon there at the top of the cliffs. Smith is thankful for the professional atmosphere fostered by the showrunners and producers which permitted him to be a part of those creative solutions. Theres an openness to solutions presented in service of the story. Theres a healthy openness to improvements regardless of the source. I think that creates an environment thats a lot of fun to work in. Its very busy, but its also incredibly rewarding. A wide shot from Season Two (Credit: Amazon).A Philosophy for Visual EffectsSmiths overall approach to the visual effects of Middle-earth has been greatly influenced by his work with The Rings of Power season one producer Ron Ames and their colleagues. Smith understands that there are common misconceptions about what visual effects experts bring to the process of filmmaking, alluding to the laughter among crew members that tends to arise when on-set visual effects teams bring out the gray reference balls utilized for computer-graphics shots. When were doing the visual effects, or were doing previz [previsualization], were making the movie. Were not delaying things with previz, were making the movie, Smith asserts, describing visual effects as no longer being a process that you stamp on at the end, like a postage stamp being affixed to an envelope. Visual effects is the pulp thats tying the paper together. Its touching everything, its influenced by everything, and it really determines so many different ways that you can or cant tell the story.Serving as The Rings of Powers senior visual effects supervisor has made Smith excited for what is on the horizon for his profession. This project has been eye-opening to me. Its incredible the things that we can do and add to make the story more fun. Theres a bright future in front of us where were going to see the lines [between the story and visual effects] blur more and more, and thats actually a good thing. I think people are going to understand that visual effects are there, and that its okay [laughs], Smith assures. Visual effects do exist, and its alright, everyone.Centering his thoughts on the step in the visual effects process that he hopes will become more widely embraced, Smith circles back to the writhing creature that emerged from Saurons corpse in the season two premieres flashback sequence. Stressing that the use of reference by visual effects artists is not just for help-in-case-you-get-stuck situations, Smith theorizes that the right reference can elevate the efficacy of a visual effects teams sheer talent. And Ill tell you what, ILM has talent. We have the top talent in the world. But, what Ive learned is, even with that top talent, if you spend time and effort getting the right reference, then your talent is picking up the baton at maybe 90% and having to carry it to 100%, instead of picking it up at 20% and having to carry it, gasping for air, to 90 or 95%. Thats the truth of it.Smiths outlook on his duty to The Rings of Power and its fans is encapsulated in his final remarks. These projects [have] finite time schedules, and visual effects is a finite resource. Thats where I see my job. I want to make sure that every visual effects artist thats on the show is really putting pixels on the screen that matter, that are telling the story, Smith concludes. I think that my value is trying to emulate Tolkien in laying the best foundation of reality for our audience that we can. Finding as much in our world to resonate with Middle-earth as we possibly can, just like he did with his writing. When he wrote about his big monsters, sometimes its just a spider thats big. I think weve been able to have some fun with that, too. We have one creature in the mud here that was done by ILM in the second season that is along those lines. I think some people will recognize the shared DNA with a small but brutal predator from here on Earth when they see it, and the result is not something Id want to encounter. We try to learn fromthe way that [Tolkien] approached things and take that on as a mantra.Jay Stobie (he/him) is a writer, author, and consultant who has contributed articles to ILM.com, Skysound.com, Star Wars Insider, StarWars.com, Star Trek Explorer, Star Trek Magazine, and StarTrek.com. Jay loves sci-fi, fantasy, and film, and you can learn more about him by visitingJayStobie.com or finding him on Twitter, Instagram, and other social media platforms at @StobiesGalaxy.