NVIDIAs Simon Yuan: Facing the Future of AI
In this Fxpodcast episode, we explore a topic that captures the imagination and attention of creatives worldwide: building your own generative AI tools and pipelines. Simon Yuen is director of graphics and AI at NVIDIA where he leads the digital human efforts to develop new character technology and deep learning-based solutions that allow new and more efficient ways of creating high-quality digital characters. Before NVIDIA, Simon spent more than 21 years in the visual effects industry, in both the art and technology sides of the problem at many studios, including Method Studios, Digital Domain, Sony Pictures Imageworks, DreamWorks, Blizzard Entertainment, and others, building teams and technologies that push the envelope of photorealistic digital character creation.Generative AI.This fxpodcast is not sponsored, but is based on research done for the new Field Guide to Generative AI. fxguides Mike Seymour was commissioned by NVIDIA to unpack the impact of generative AI on the media and entertainment industries, offering practical applications, ethical considerations, and a roadmap for the future.The Field Guide is free and can be downloaded here: Field Guide to Generative AI. In M&E, generative AI has proven itself a powerful tool for boosting productivity and creative exploration. But it is not a magic button that does everything. Its a companion, not a replacement. AI lacks the empathy, cultural intuition, and nuanced understanding of a storys uniqueness that only humans bring to the table. But when generative AI is paired with VFX artists and TDs, it can accelerate pipelines and unlock new creative opportunities.Digital Human powered by NVIDIAThe Core of Generative AI: Foundation Models & NIMs.NVIDIA Inference Microservices (NIMs) and foundation models are the building blocks of alot of modern AI workflows, and they are at the heart of many new generative AI solutions.Foundation models are large-scale, pre-trained neural networks that can tackle broad categories of problems. Think of them as AI generalists, adaptable to specific tasks through fine-tuning with additional data. For example, you might start with a foundation model capable of understanding natural language (LLM) and fine-tune it to craft a conversational agent that your facility can use to help on board new employees.While building these models from scratch is resource-intensive and time-consuming, fine-tuning them for your specific application is relatively straightforwardand NVIDIA has made this process quite accessible.NVIDIA NIMs.NIMS or microservices simplify the deployment of foundation models. Whether on the cloud, in a data center, or even on the desktop. NIMs streamline the process but also ensuring data security. They make it easy to create tailored generative AI solutions for facilitys or project needs. For instance, NVIDIAs latest OpenUSD NIMs allow developers to integrate generative AI copilots into USD workflows, enhancing efficiency in 3D content creation.JamesBringing Digital Humans to Life with NVIDIA ACEOne of the most interesting applications of NIMs is in crafting lifelike digital humans. NVIDIAs ACE (Avatar Cloud Engine) exemplifies this capability. With ACE, developers and TDs can design interactive digital humans and avatars that respond in real-time with authentic animations, speech, and emotions.A standout example is James, a virtual assistant powered by NVIDIA ACE. He is an interactive digital human, a communications tool powered by a selected knowledge base, ACE and animated by NVIDIAs Audio2Face. James showcases how generative AI and digital human technologies converge, providing tools for telepresence, interactive storytelling, or even live character performances. This is more than just a visual upgradeits a way to enhance emotional connections in digital media.Generative AI: Empowering Creativity, Not Replacing ItAs we as an industry adopt and explore these tools, its essential to keep a balanced perspective. Generative AI isnt here to replace human creativity we need to use it to amplify it. Ai can enable teams to iterate faster, experiment, and focus on the storytelling that truly resonate with audiences. Central to this is respecting artists rights, having providence of training data and maintaining data security.From fine-tuning a foundation model to integrating NIM-powered workflows, building your own generative AI workflow involves leveraging technology to empower your project. With tools like NVIDIAs foundation models and ACE, the possibilities are immense, but the responsibility to use them thoughtfully is equally crucial.