A committee of experts from top U.S. medical centers and research institutes is harnessing NVIDIA-powered federated learning to evaluate the impact of federated learning and AI-assisted annotation to train AI models for tumor segmentation.Federated learning is a technique for developing more accurate, generalizable AI models trained on data across diverse data sources without mitigating data security or privacy. It allows several organizations to collaborate on the development of an AI model without sensitive data ever leaving their servers.Due to privacy and data management constraints, its growing more and more complicated to share data from site to site and aggregate it in one place and imaging AI is developing faster than research institutes can set up data-sharing contracts, said John Garrett, associate professor of radiology at the University of WisconsinMadison. Adopting federated learning to build and test models at multiple sites at once is the only way, practically speaking, to keep up. Its an indispensable tool.Garrett is part of the Society for Imaging Informatics and Medicine (SIIM) Machine Learning Tools and Research Subcommittee, a group of clinicians, researchers and engineers that aims to advance the development and application of AI for medical imaging. NVIDIA is a member of SIIM, and has been collaborating with the committee on federated learning projects since 2019.Federated learning techniques allow enhanced data privacy and security in compliance with privacy regulations like GDPR, HIPAA and others, said committee chair Khaled Younis. In addition, we see improved model accuracy and generalization.To support their latest project, the team including collaborators from Case Western, Georgetown University, the Mayo Clinic, the University of California, San Diego, the University of Florida and Vanderbilt University tapped NVIDIA FLARE (NVFlare), an open-source framework that includes robust security features, advanced privacy protection techniques and a flexible system architecture.Through the NVIDIA Academic Grant Program, the committee received four NVIDIA RTX A5000 GPUs, which were distributed across participating research institutes to set up their workstations for federated learning. Additional collaborators used NVIDIA GPUs in the cloud and in on-premises servers, highlighting the flexibility of NVFLare.Cracking the Code for Federated LearningEach of six participating medical centers provided data from around 50 medical imaging studies for the project, focused on renal cell carcinoma, a kind of kidney cancer.The idea with federated learning is that during training we exchange the model rather than exchange the data, said Yuankai Huo, assistant professor of computer science and director of the Biomedical Data Representation and Learning Lab at Vanderbilt University.In a federated learning framework, an initial global model broadcasts model parameters to client servers. Each server uses those parameters to set up a local version of the model thats trained on the organizations proprietary data. Then, updated parameters from each of the local models are sent back to the global model, where theyre aggregated to produce a new global model. The cycle repeats until the models predictions no longer improve with each training round.The group experimented with model architectures and hyperparameters to optimize for training speed, accuracy and the number of imaging studies required to train the model to the desired level of precision.AI-Assisted Annotation With NVIDIA MONAIIn the first phase of the project, the training data used for the model was labeled manually. For the next phase, the team is using NVIDIA MONAI for AI-assisted annotation to evaluate how model performance differs with training data segmented with the help of AI compared to traditional annotation methods.The biggest struggle with federated learning activities is typically that the data at different sites is not tremendously uniform. People use different imaging equipment, have different protocols and just label their data differently, said Garrett. By training the federated learning model a second time with the addition of MONAI, we aim to find if that improves overall annotation accuracy.The team is using MONAI Label, an image-labeling tool that enables users to develop custom AI annotation apps, reducing the time and effort needed to create new datasets. Experts will validate and refine the AI-generated segmentations before theyre used for model training.Data for both the manual and AI-assisted annotation phases is hosted on Flywheel, a leading medical imaging data and AI platform that has integrated NVIDIA MONAI into its offerings.Once the project is complete, the team plans to publish their methodology, annotated datasets and pretrained model to support future work.Were interested in not just exploring these tools, Garrett said, but also publishing our work so others can learn and use these tools throughout the medical field.Apply for an NVIDIA Academic GrantThe NVIDIA Academic Grant Program advances academic research by providing world-class computing access and resources to researchers. Applications are now open for full-time faculty members at accredited academic institutions who are using NVIDIA technology to accelerate projects in simulation and modeling, generative AI and large language models.Future application cycles will focus on projects in data science, graphics and vision, and edge AI including federated learning.

All the electricity that powers NVIDIAs global operations will come from renewable sources by the end of January.Its the right fuel for the companys mission: to help customers and partners harness AI and accelerated computing for sustainable growth.This week, world leaders gather in New York City to mark Climate Week 2024. Its a good time to take stock of recent gains in energy efficiency in fields from manufacturing and cloud computing to healthcare and beyond.Foxconn Saves Energy With Digital TwinsFoxconn, the worlds largest electronics manufacturer, is using accelerated computing and AI to build a digital twin of a new factory in Mexico, where it will train its robots and define production processes.Our digital twin will guide us to new levels of automation and industrial efficiency, saving time, cost and energy, said Young Liu, chairman of Foxconn, which estimates a 30% annual energy savings.Its the latest company to employ software from the Siemens Xcelerator portfolio including Teamcenter and NVIDIA Omniverse, a platform for developing 3D workflows and applications based on OpenUSD.We will revolutionize how products and experiences are designed, manufactured and serviced, said Roland Busch, president and CEO of Siemens AG. In collaboration with NVIDIA, we will bring accelerated computing, generative AI and Omniverse integration across the Siemens Xcelerator portfolio.Siemens and NVIDIA executives will talk about their sustainability efforts in a Sept. 25 Climate Week event moderated by TIME magazine.Cloud Services Increase EfficiencyCloud computing services are also racking up advances in energy efficiency.An e-commerce website uses NVIDIA AI to connect hundreds of millions of shoppers a day to the products they need. After migrating from CPUs to GPUs, it achieved significantly lower latency with a 33x speedup and nearly 12x energy-efficiency improvement.A popular video conferencing application captions several hundred thousand virtual meetings an hour. By switching from CPUs to GPUs, throughput increased from just three to 200 queries per second a 66x speedup and 25x energy-efficiency improvement.To meet sustainability goals, service providers and their customers are increasingly turning to renewable energy sources.For example, Equinix is the first global data center provider to publish targets to become climate neutral using 100% renewable energy by 2030. Currently, renewables make up 96% of the energy it uses, giving customers access to technologies such as NVIDIA DGX systems for sustainable, private AI.The DGX systems use NVIDIA H100 Tensor Core GPUs, which offer up to 4x more energy efficiency than previous models. Equinix employs advanced liquid cooling to reduce power and water consumption.The evolution of the data center will be discussed at a Climate Week event hosted by NVIDIA and Crusoe Energy Systems, a company that builds and operates clean computing infrastructure.Decoding Cancer While Saving EnergyHealthcare companies are adopting AI and accelerated computing to help them understand and treat diseases.For instance, the Wellcome Sanger Institute runs one of the largest genome sequencing facilities in the world. Its cancer program sequences and analyzes tens of thousands of cancer samples.Using NVIDIA Parabricks software on NVIDIA DGX systems reduces its run times by 1.6x, capital costs by 24x and energy consumption by up to 42x. That amounts to potential savings of $1 million and 1,000 megawatt-hours a year.The latest version of Parabricks, released in mid-September, can further enhance such work.The Sanger Institute handles hundreds of thousands of samples annually, said Jingwei Wang, a principal software developer at Wellcome Sanger in a profile describing its work. With NVIDIA GPUs and Parabricks accelerated genomic analysis software, we will save considerable time, cost and energy analyzing them.Expanding the Sustainability EcosystemIn addition to enabling efficiency gains for customers, NVIDIA is nurturing a broad ecosystem focused on sustainability.For example, a collaboration with the United Nations announced last year has already trained more than 14,000 data scientists on how to design AI models for early flood detection.Entrepreneurs increasingly see opportunities here, too.Sustainable Futures, a member of the NVIDIA Inception program, now includes more than a thousand startups pioneering new paths to energy efficiency. They apply a mix of AI, accelerated computing and open platforms like NVIDIA Earth-2 to speed climate and weather predictions.Savings Span Global IndustriesThese examples provide a taste of whats happening and whats to come.The generative AI revolution is poised to impact every industry and enable a new era of productivity and sustainability by unlocking efficiencies and resource optimization across sectors, said NVIDIA founder and CEO Jensen Huang in the companys latest sustainability report.Its a reality independent analysts recognize.Even if the predictions that data centers will soon account for 4% of global energy consumption become a reality, AI is having a major impact on reducing the remaining 96% of energy consumption, said a report from Lisbon Council Research, a nonprofit formed in 2003 that studies economic and social issues.45,000x Efficiency Gain in AI InferenceNVIDIA works continuously to expand and optimize its full stack of technologies to enable greater efficiencies.In the last eight years, the NVIDIA platform has gotten 45,000x more energy efficient when processing large language models. If the efficiency of cars improved that much, cars would get 280,000 miles per gallon enough to drive to the moon on less than a gallon of gas.Speedups over CPUs from 20x to 160x and their associated energy savings are available across the gamut of workloads from data processing to computer vision the NVIDIA accelerated computing platform serves. Thats why accelerated computing is often called sustainable computing.Expanding Climate Weeks AgendaRallying support for sustainability, NVIDIA and partners are taking part in a handful of other Climate Week events.They include a Climate AI Summit with Salesforce and a Climate Science Fair with about 30 startups, many using NVIDIA technologies. The Science Fair will give attendees an up-close look at how Earth-2 enables simulation on a planetary scale.Its another small step toward helping people understand how to use AI and accelerated computing to address climate change.Accelerated computing is how to meet the massive demand for computing power sustainably and cost-effectively, Huang said.Learn more about how NVIDIAs technologies support corporate sustainability.

In a surprise appearance at T-Mobiles Capital Markets Day, NVIDIA founder and CEO Jensen Huang shared a bold vision for the future of telecommunications.Weve fused signal processing and AI, Huang declared during a fireside chat with T-Mobile CEO Mike Sievert, speaking to an audience of press, analysts and investors. This is going to be a great new growth opportunity for the telecommunications industry.Huangs remarks came alongside NVIDIAs announcement of its groundbreaking AI Aerial platform, which promises to reshape wireless networks by integrating AI and radio access networks, AI-RAN.The platform is designed to optimize network performance, efficiency and new revenue potential, such as AI-computing-as-a-service during periods when network infrastructure is underutilized, maximizing the return on assets.During the conversation, Huang emphasized the importance of AI in shaping the future of telecommunications, particularly highlighting the role of AI-RAN in optimizing and scaling network performance.Fusing radio computing and AI computing into one architecture allows companies to apply AI models to optimize signal quality across diverse environments, Huang explained.He emphasized that this fusion would lead to improved network efficiency and new growth opportunities for the telecommunications industry,We could teach these AI models how to optimize signal quality in hundreds of thousands of virtual cities, Huang said.AI-RAN aligns with NVIDIAs broader vision to make AI an integral part of network infrastructure, enabling telecommunications providers to unlock new revenue streams and deliver enhanced experiences through generative AI, robotics and autonomous technologies.Huang underscored the synergies between NVIDIA and T-Mobile, particularly their collaboration on the newly announced AI-RAN Innovation Center, as co-authors of transformation. The AI-RAN Innovation Center, developed with T-Mobile, Ericsson and Nokia, is set to accelerate the commercialization of AI-RAN technologies.Every radio operates in a unique and constantly changing world environment. This is where deep reinforcement learning algorithms embedded into radio signal processing make complex computations simpler with AI to help deliver a customer-centric network experience.Sievert emphasized how virtualizing RAN into the cloud will create new business opportunities. He explained that AI workloads will increasingly require compute power located close to the customer, leveraging underutilized network resources.Huang also highlighted the crucial role AI will play in making networks more energy-efficient, emphasizing the need for sustainable technology as the demand for data and connectivity grows.We have to use AI to reduce energy consumption, Huang said. Everything that we accelerate, everything that we teach an AI model to do [we] will do a lot more energy efficiently.As Huang explained, by simulating AI models in virtual environments with accurate physics and then emulating them in the real world, NVIDIA maximizes energy efficiency. This approach underpins the NVIDIA AI Aerial suite of platforms for designing, training and deploying AI-driven cellular networks for AI.With NVIDIA AI Aerial now supporting a growing ecosystem of partners this collaboration marks a milestone in the telecom industrys journey toward a future powered by AI.

GeForce NOW makes gamers fantasies a reality by bringing top titles to the cloud. This week, the award-winning FINAL FANTASY XVI is available for members to stream at launch.Catch 11 bit studios city-survival game Frostpunk 2 before it launches with early access on GeForce NOW.Its part of seven new games joining the expansive GeForce NOW library.Eikonic AdventureFeel like a king streaming from the cloud with ultra-low latency.Dive into the richly crafted world of Valisthea in FINAL FANTASY XVI, the latest mainline entry in Square Enixs renowned role-playing game series. This epic action RPG unfolds in a once-glorious land blessed by the light of the Mothercrystals thats now slowly succumbing to the mysterious Blight.Play as Clive Rosfield as he navigates a dark fantasy filled with political intrigue and magical battles against Eikons, powerful manifestations of elemental forces. Eikons are wielded by Dominants, individuals who possess incredible strength and abilities.Featuring exhilarating real-time combat, the game allows players to unleash devastating moves by chaining together attack combos with seamless fluidity. With its gripping story, stunning visuals and intense gameplay, FINAL FANTASY XVI offers an unforgettable experience for new and longtime fans of the series alike.Streaming the game with GeForce NOW, explore the captivating world of Valisthea without needing to worry about hardware specs or download space. Ultimate members can even stream at up to 4K resolution to experience the full visual splendor of the game, making every battle and cinematic moment even more engaging and memorable. Series newcomers can try the demo on GeForce NOW before purchasing the full game.Winter Is ComingChill out in the cloud.The sequel to the hit title Frostpunk will be available for members to stream at launch on Friday, Sept. 20. Members can dive into the frigid dystopia now with Advanced Access to Frostpunk 2 on GeForce NOW for those that preorder the Deluxe Edition on Steam and Xbox.Frostpunk 2 builds on the critically acclaimed original, challenging newcomers and veterans of the series to lead a resource-starved civilization in a world locked in an eternal winter. The sequel brings more complex and expanded city-building mechanics and intricate political scenarios. Make tough choices to keep citizens alive and somewhat happy while navigating the harsh realities of this frozen world.Experience Frostpunk 2 in all its icy glory on GeForce NOW and brave the eternal winter without breaking a sweat over hardware specs. Chill out with a Priority or Ultimate membership for longer gaming sessions over free users to develop intricate, sprawling cities.Lets Get GreedyHelp the Payday Gang take the system down.Dive into the high-stakes world of PAYDAY 3 as the latest downloadable content and final Year 1 chapter of the game, Fear and Greed, is available for members to stream now. Drop into the heart of Wall Street to pull off the ultimate insider trading scheme, uncover conspiracies and lead the Payday Gang to hit Concords stocks and bring it all crumbling to the ground.Members can look for the following games available to stream in the cloud this week:Frostpunk 2 (New release on Steam and Xbox available on PC Game Pass, Sept. 20)FINAL FANTASY XVI (New release on Steam and Epic Games Store, Sept. 17)The Plucky Squire (New release on Steam, Sept. 17)Dead Rising Deluxe Remaster (New release on Steam, Sept. 18)Romancing SaGa 2: Revenge of the Seven (Demo releases on Steam, Sept. 18)The Legend of Heroes: Trails Through Daybreak (Steam)REKA (Steam)What are you planning to play this weekend? Let us know on X or in the comments below.Ever feel like you're just one challenge away from a total game-changer? or is it just us living in an action-packed fantasy? NVIDIA GeForce NOW (@NVIDIAGFN) September 18, 2024

Companies in the US spend $15bn annually on talent acquisition. The most important metric in recruitment advertising is the conversion from the paid click on the job platform to the application the employer receives. Industry-wide, apply conversion is just 5%. Redirection of the candidate from the job platform to the company site is the biggest cause of abandonment; this step has a 70% bounce rate. In this episode of NVIDIAs AI Podcast, host Noah Kravitz speaks with Mikhil Raja, Cofounder and CEO of SonicJobs, about how they have built AI Agents to enable candidates to complete applications directly on job platforms, without redirection, boosting completion rates to 26% from 5%. Raja delves deep into SonicJobs cutting-edge technology, which merges traditional AI with large language models (LLMs) to understand and interact with job application web flows. He also emphasizes the importance of fine-tuning foundational models to achieve more impactful and scalable innovations.SonicJobs is a member of the NVIDIA Inception program for startups.The AI Podcast SonicJobs CEO Mikhil Raja on Using AI to Connect Job Seekers With EmployersTime Stamps1:19: Why applying for a job remains a Web 1.0 experience and how SonicJobs AI Agents are changing this6:06: Explanation of SonicJobs technology and the benefits to users and companies9:03: The evolution of AI Agents from AutoGPT to Verticalized B2B solutions11:33: How SonicJobs realized the approach it should take with Agentic AI15:18: Scaling SonicJobs AI Agent and the adaptive learning flywheel17:45: Raja discusses the need for accuracy including fine-tuning foundational models20:45: Framework for how SonicJobs Verticalized AI Agent solution can be applied to further Verticals23:23: Advice Raja would give to a company thats currently trying to hireYou Might Also LikeHow Georgia Techs AI Makerspace Is Preparing the Future Workforce for AI Ep. 229AI is set to transform the workforce and the Georgia Institute of Technologys new AI Makerspace is helping tens of thousands of students get ahead of the curve. Arijit Raychowdhury, a professor and Steve W. Cedex school chair of electrical engineering at Georgia Techs college of engineering, shares about the schools supercomputer hub, which provides students with the computing resources to reinforce their coursework and gain hands-on experience with AI.How Roblox Uses Generative AI to Enhance User Experiences Ep. 227Roblox is a colorful online platform that aims to reimagine how people come together now that vision is being augmented by generative AI. Anupam Singh, vice president of AI and growth engineering at Roblox, speaks about how the company uses the technology to enhance virtual experiences with features such as automated chat filters and real-time text translation.Performance AI: Insights from Arthurs Adam Wenchel Ep. 221Adam Wenchel, cofounder and CEO of Arthur, explains how the company enhances the performance of AI systems across various metrics like accuracy, explainability, and fairness. He also shares insights into the challenges and opportunities of deploying generative AI.Media.Monks Lewis Smithingham on Enhancing Media and Marketing With AI Ep. 222Meet Media.Monks Wormhole, an alien-like, conversational robot with a quirky personality and the ability to offer keen marketing expertise. Lewis Smithingham, senior vice president of innovation and special ops at Media.Monks, a global marketing and advertising company, discusses the creation of Wormhole and AIs potential to enhance media and entertainment.Subscribe to the AI PodcastGet the AI Podcast through iTunes, Google Play, Amazon Music, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn.Make the AI Podcast better: Have a few minutes to spare? Fill out this listener survey.

Telecommunications providers are transforming beyond voice and data services with an AI computing infrastructure to optimize wireless networks and serve the next-generation needs of generative AI on mobile, robots, autonomous vehicles, smart factories, 5G and much more.Launched today, NVIDIA AI Aerial is a suite of accelerated computing software and hardware for designing, simulating, training and deploying AI radio access network technology (AI-RAN) for wireless networks in the AI era.The platform will become a critical foundation to allow network optimization at scale to serve the demands of a host of new services. This will provide significant savings in total cost of ownership and open telecom operators to new revenue opportunities for enterprise and consumer services.NVIDIA AI Aerial enables telecommunications service providers to support teleoperations for manufacturing robots and autonomous vehicles, computer vision in manufacturing and agriculture, logistics, generative AI-driven co-pilots and personal assistants, emerging spatial computing applications, robotic surgery, 3D collaboration, and 5G and 6G advances.Driving Networks of the Future With AI-RANNVIDIA AI Aerial is the worlds first AI-RAN platform capable of hosting generative AI and RAN traffic, as well as integrating AI into network optimization.AI-RAN offers high-performance and energy-efficient software-defined RAN, improved network experience and new revenue opportunities with edge AI applications to host internal and third-party generative AI applications.AI-RAN is foundational to the multipurpose networks of tomorrow that rely on AI-powered telecommunications capabilities.Harnessing NVIDIA AI Aerial for Telecom IndustryThe NVIDIA AI Aerial platform offers access to a full suite of capabilities, including a high-performance, software-defined RAN along with training, simulation and inference so that telecom operators can participate at any stage of development to deployment for next-generation wireless networks.Some capabilities in the NVIDIA AI Aerial platform include:NVIDIA Aerial CUDA-Accelerated RAN includes software libraries to enable partners to develop and deploy high-performance virtualized RAN workloads on NVIDIA-accelerated compute platforms.NVIDIA Aerial AI Radio Frameworks include PyTorch- and TensorFlow-based software libraries to develop and train models for improving spectral efficiency and adding new capabilities to 5G and 6G radio signal processing. This also includes NVIDIA Sionna, a link-level simulator that provides development and training of neural network-based 5G and 6G radio algorithms.NVIDIA Aerial Omniverse Digital Twin (AODT) is a system-level network digital twin development platform. AODT enables physically accurate simulations of wireless systems from a single base station to a comprehensive network with a large number of base stations covering an entire city. It incorporates software-defined RAN (Aerial-CUDA Accelerated RAN) and user-equipment simulators, along with realistic terrain and object properties of the physical world.NVIDIA AI Aerial and AI RAN Innovation CenterNVIDIA is collaborating with T-Mobile, Ericsson and Nokia to accelerate the commercialization of AI-RAN with the establishment of the AI-RAN Innovation Center.The center will tap into key capabilities of the NVIDIA AI Aerial platform. The collaboration is focused on bringing RAN and AI innovation closer together to deliver transformational network experiences for customers through the development of AI-RAN [link to the TMUS PR release on AI-RAN].AI-RAN is set to revolutionize the telecom industry, and the opening of the AI-RAN Innovation Center will help to take us on this journey by driving industry collaboration, said Tommi Uitto, president of Mobile Networks at Nokia. By bringing together leading companies in the telecom and AI industries, we can unlock the full potential of AI in our networks, improving performance, reducing costs and creating new opportunities for our customers. We are confident that AI-RAN will be a key driver of innovation in the future, and we are excited to be part of this revolution together with NVIDIA.Ericsson has invested in our AI-RAN solution, allowing communications service providers to deploy portable RAN software running across multiple platforms. We are now evaluating the performance and cost of NVIDIA accelerated computing in this context, said Fredrik Jejdling, EVP and head of Business Area Networks at EricssonNVIDIA AI Aerial EcosystemKey partners of the growing ecosystem of NVIDIA AI Aerial include Softbank and Fujitsu.Ansys and Keysight are using NVIDIA Aerial Omniverse Digital Twin for testing and simulation systems, while partners and academia such as Deepsig, ETH-Zurich, Northeastern University and Samsung are collaborating on 6G research and NVIDIA Aerial AI Radio Frameworks.Cloud stack software providers such as Aarna Networks, Canonical, Red Hat and Wind River; networking stack providers like Arrcus network and server infrastructure providers like Dell Technologies, Hewlett Packard Enterprise and Supermicro are key partners for NVIDIA AI Aerial. Edge Solution providers like Vapor.io, and system integrators like World Wide Technology, with its AI Proving Ground, [insert link to their blog] are accelerating decision-making for AI solutions.Learn more about NVIDIA AI Aerial

A new AI innovation hub for developers across Tunisia launched today in Novation City, a technology park thats designed to cultivate a vibrant, innovation ecosystem in mechatronics an industry encompassing IT, mechanics and electronics and to foster synergy between education, research and industry in the North African country.Built in collaboration with the NVIDIA Deep Learning Institute (DLI), the hub offers the training, technologies and business networks needed to help drive AI adoption across the continent.The hubs launch is part of NVIDIAs efforts to train 100,000 developers across Africa through the DLI over the next three years a goal thats about a quarter complete today.Located in Sousse a coastal city in central Tunisia thats surrounded by universities, startups and other organizations with a strong focus on STEM and AI the hub includes complimentary access to NVIDIA DLI courses on topics such as generative AI, accelerated computing and data science.Through its AI, industry 4.0 and smart transport centers of excellence, Novation City provides cutting-edge resources and access to NVIDIA DGX infrastructure for AI startups and researchers. Novation City also hosts organized activities to drive ecosystem growth, such as hackathons and specialized training sessions. Its all brought together in this new environment conducive to AI learning, experimentation and deployment.Novation City has launched several key AI initiatives to strengthen the ecosystem, with NVIDIAs support being instrumental in empowering AI startups and advancing AI skills, said Anas Rochdi, chief innovation officer at Novation City. This year, we deployed Tunisias first NVIDIA DGX system and launched major academic initiatives in collaboration with the NVIDIA Deep Learning Institute, aiming to train more than 1,000 developers in one year.Novation City also runs several startup accelerator programs, and more than 10 participating companies are members of NVIDIA Inception, a free program that nurtures cutting-edge startups.Tunisia Drives Innovation in STEM and AI EducationTunisias education system has traditionally emphasized STEM, especially mathematics and the sciences, according to Wei Xiao, who leads NVIDIAs developer relations team for startups, enterprises and universities in the Middle East and Africa.Tunisia has a rich history of valuing knowledge and scholarship, dating back to ancient Carthage and the Islamic Golden Age, Xiao said. And the nations curriculum is rigorous, creating a solid foundation for advanced studies in STEM fields.This has made the country well-situated to serve as a gateway between Europe, Africa and the Middle East (EMEA) a thriving ecosystem for innovation, entrepreneurship and research. Novation City and NVIDIA aim to bolster the ecosystem even further through the new AI innovation hub.Fostering collaboration between academia, industry and government, the initiative is funded by French and German development agencies, the Tunisian government, the World Bank and the European Union, as well as several enterprises.And the hub comes at a time when Tunisia has adopted a national strategy for AI and digitalization which includes promoting AI education and research as part of a broader vision to position the nation as a digital leader in Africa. For example, the University of Tunis this month launched the nations first public institute specializing in AI.Partners Foster Sovereign AI in Tunisia and BeyondA key part of sovereign AI is a nations ability to produce artificial intelligence using its own workforce along with its own infrastructure, data and business networks. Free DLI training offered through Tunisias AI innovation hub is poised to enable just that, helping upskill the next generation of African AI experts.Plus, Novation City already offers a wide range of facilities designed to support technological and scientific advancement.In February, Novation City deployed an NVIDIA DGX system, among the first in Africa, that has empowered about 30 startups across the continent in climate AI, transportation, manufacturing, agtech and other industries to develop accelerated computing-based solutions.In addition, ESPRIT University a specialized university in Tunisia with more than 10,000 engineering students boasts nine NVIDIA DLI ambassadors who are delivering training to students and contributing to the broader tech ecosystem across the country. This makes ESPRIT University one of the most active DLI organizations across EMEA.Since 2018, ESPRIT has been tapping into DLI to advance AI education. The university has also acquired an NVIDIA DGX system to support research and product development.NVIDIA has planned similar AI education initiatives in Kenya and Nigeria to further upskill and enhance African technology ecosystems.Learn more about the NVIDIA Deep Learning Institute.

At TwitchCon a global convention for the Twitch livestreaming platformlivestreamers and content creators this week can experience the latest technologies for accelerating creative workflows and improving video quality.That includes the beta release of Twitch Enhanced Broadcasting support for HEVC when using the NVIDIA encoder.Content creators can also use the NVIDIA Broadcast app, eighth-generation NVIDIA NVENCand RTX-powered optimizations in streaming and video editing apps to enhance their productions.Plus, the September NVIDIA Studio Driver, designed to optimize creative apps, is now ready for download. Studio Drivers undergo extensive testing to ensure seamless compatibility while enhancing features, automating processes and accelerating workflows.Twitch Enhanced Broadcasting With HEVCThe tradeoff between higher-resolution video quality and reliable streaming is a common issue livestreamers struggle with.Higher-quality video provides more enjoyable viewing experiences but can cause streams to buffer for viewers with lower bandwidth or older devices. Streaming lower-bitrate video allows more people to watch content seamlessly but introduces artifacts that can interfere with viewing quality.To address this issue, NVIDIA and Twitch collaborated to develop Twitch Enhanced Broadcasting. The feature adds the capability to send multiple streams different versions of encoded video with different resolutions or bitrates directly from NVIDIA GeForce RTX-equipped PCs or NVIDIA RTX workstations to deliver the highest-quality video a viewers internet connection can handle.Twitch supports HEVC (H.265) in the Enhanced Broadcasting closed beta. With the NVIDIA encoder, Twitch streamers get 25% improved efficiency and quality over H.264.This means that video will look as if it were being streamed with 25% more bitrate in higher quality and with reduced artifacts or encoding errors. The feature is ideal for streaming fast-paced gameplay, enabling cleaner, sharper video with minimal lag.Because all stream versions are generated with a dedicated hardware encoder on GeForce RTX GPUs, the rest of the systems GPU and CPU are free to focus on running games more smoothly to maximize performance.Learn how to get started on twitch.com.AI-Enhanced Microphones and WebcamsStreaming is easier than ever with NVIDIA technologies.For starters, PC performance and video quality are incredibly high quality thanks to NVIDIAs dedicated encoder. And, NVIDIA GPUs include Tensor Cores that efficiently run AI.Livestreamers can use AI to enhance their hardware peripherals and devices, which is especially helpful for those who havent had the time or resources to assemble extensive audio and video setups.NVIDIA Broadcast transforms any home office or dorm room into a home studio without the need to purchase specialized equipment. Its AI-powered features include Noise and Echo Removal for microphones, and Virtual Background, Auto Frame, Video Noise Removal and Eye Contact for cameras.Livestreamers can download the Broadcast app or access its effects across popular creative apps, including Corsair iCUE, Elgato Camera Hub, OBS, Streamlabs, VTube Studio and Wave Link.Spotlight the HighlightsGeForce RTX GPUs make it lightning-fast to edit and enhance video footage on the most popular video editing apps, from Adobe Premiere Pro to CapCut Pro.Streamers can use AI-powered, RTX-accelerated features like Enhance Speech to remove noise and improve the quality of dialogue clips; Auto Reframe to automatically size social media videos; and Scene Edit Detection to break up long videos, like B-roll stringouts, into individual clips.NVIDIA encoders help turbocharge the export process. For those looking for extreme performance, the GeForce RTX 4070 Ti GPU and up come equipped with dual encoders that can be used in parallel to halve export times on apps like CapCut, the most widely used video editing app on TikTok.Clearer, Sharper Viewing Experiences With RTX VideoNVIDIA RTX Video available exclusively for NVIDIA and GeForce RTX GPU owners can turn any online and native video into pristine 4K high dynamic range (HDR) content with two technologies: Video Super Resolution and Video HDR.RTX Video Super Resolution de-artifacts and upscales streamed video to remove errors that occur during encoding or transport, then runs an AI super-resolution effect. The result is cleaner, sharper video thats ideal for streaming on platforms like YouTube and Twitch.Many users have HDR displays, but there isnt much HDR content online. RTX Video HDR addresses this by turning any standard dynamic range (SDR) video into HDR10 quality that delivers a wider range of brights and darks and makes visuals more vibrant and colorful. This feature is especially helpful when watching dark-lit scenes in video games.RTX Video HDR requires an RTX GPU connected to an HDR10-compatible monitor or TV. For more information, see the RTX Video FAQ.Check out TwitchCon taking place in San Diego and online from Sept. 20-22 for the latest streaming updates.

Rise and shine Capcoms latest action-adventure game, Dead Rising Deluxe Remaster, heads to the cloud at launch next week.Its part of nine new titles joining the extensive GeForce NOW library.Dead Rising Coming SoonFrom the ground to the cloud.Dead Rising Deluxe Remaster returns with modern graphics. More than just a remaster, this Deluxe Remaster is a full graphical overhaul of the first game in the zombie-slaughtering action series Dead Rising. The remaster has also been fully voiced, supports auto-saves and has various other quality-of-life features.One day, the peaceful town of Willamette, Colorado, found itself put under quarantine by the U.S. army. Frank West, a freelance journalist, smells a scoop and finds his way into the only shopping mall in town. Unfortunately, the mall has turned into a living hell, crawling with countless zombies. Help will arrive in 72 hours, so its up to him to find out the truth behind this incident before its too late.Witness the unmatched mayhem and freedom when Dead Rising Deluxe Remaster launches on Wednesday, Sept. 19. Stream it with a GeForce NOW Priority or Ultimate membership for longer gaming sessions and higher frame rates.New Games to Drive You WildVroom, vroom.Test Drive Unlimited Solar Crown, a new open-world racing game from KT Racing and Nacon, is now available for members to stream. Explore a fully recreated Hong Kong Island while taking to the road behind the wheels of exceptional cars and living the ultimate life of luxury. Test drive and purchase cars directly from dealerships, customize them in workshops and display them in the Solar Hotel garage. Each car offers a unique driving experience.Members can look for the following games available to stream in the cloud this week:Warhammer 40,000: Space Marine 2 (New release on Steam, Sept. 9)Test Drive Unlimited Solar Crown (New release on Steam, Sept. 12)Dawn of Defiance (Steam)Flintlock: The Siege of Dawn (Xbox, available on PC Game Pass)Fort Solis (Epic Games Store)King Arthur: Legion IX (Steam)Squirrel With a Gun (Steam)Tyranny Gold Edition (Xbox, available on Microsoft Store)XIII (Xbox, available on Microsoft Store)What are you planning to play this weekend? Let us know on X or in the comments below.Which game(s) do you think would be cool as the opposite perspective? NVIDIA GeForce NOW (@NVIDIAGFN) September 10, 2024

AI is transforming the broadcast industry by enhancing the way content is created, distributed and consumed but integrating the technology can be challenging.Launched this week in limited availability, NVIDIA Holoscan for Media is a software-defined, AI-enabled platform that helps developers easily integrate AI into their live media applications and allows media companies to run live media pipelines on the same infrastructure as AI.NVIDIA RTX AI workstations and PCs, powered by NVIDIA GPUs for real-time graphics processing and AI computing, provide an ideal foundation for developing these applications.At the IBC broadcast and media tech show in Amsterdam, NVIDIA partners including Adobe, Blackmagic Design and Topaz Labs will showcase the latest RTX AI-powered video editing tools and technologies powering live media advancements.NVIDIA Holoscan for Media: Building the Future of Live ProductionNVIDIA Holoscan for Media is an AI-enabled, software-defined platform for live media.Building a robust AI software stack for application development in live media is an intricate process that requires substantial expertise and resources.This technical complexity, coupled with the need for large amounts of high-quality data and the difficulty of scaling pilot programs to production-level performance, often prevents these initiatives from reaching full deployment. Additionally, traditional development of software is tied to dedicated hardware, further limiting innovation and making upgrades cumbersome.Addressing these challenges, NVIDIA Holoscan for Media empowers developers to create cutting-edge AI applications for live media with ease through its seamless integration with NVIDIAs extensive suite of AI software development kits (SDKs). This allows developers to easily incorporate advanced AI capabilities into their applications so they can focus on creating more sophisticated and intelligent media applications. Media companies can then seamlessly connect those applications to live video pipelines running on top of the platform.Another typical challenge in live media application development is inefficiency in deployment. Developers often find themselves needing to create separate builds for different deployment types, whether on premises, in the cloud or at the edge. This increases costs and can extend development timelines. Developers must also allocate resources to build additional infrastructure services, such as authentication and timing protocols, further straining budgets.Holoscan for Medias cloud-native architecture enables applications to run from anywhere. Applications developed for the cloud, edge or on-premises deployments can run across environments, eliminating the need for separate builds.Holoscan for Media is available on premises today, with cloud and edge deployments coming soon. The platform also includes Precision Time Protocol for audio-video synchronization in live broadcasts and Networked Media Open Specifications for seamless communication between applications simplifying the management of complex systems.Enhancing Development With RTX AI PCs and WorkstationsNVIDIA RTX AI PCs and workstations complement the potential of Holoscan for Media by offering a robust foundation for developing immersive media experiences.The CUDA ecosystem available on RTX AI PCs and workstations offers access to a vast array of NVIDIA SDKs and tools optimized for media and AI workloads. This allows developers to build applications that can seamlessly transition from workstation to deployment environments, ensuring that their creations are both robust and scalable.NVIDIA AI Enterprise offers further enhancements by putting a comprehensive suite of AI software, tools and frameworks optimized for NVIDIA GPUs into the hands of enterprise developers who require secure, stable and scalable production environments for AI applications. This enterprise-grade AI platform includes popular frameworks like TensorFlow, PyTorch and RAPIDS for streamlined deployment.Using NVIDIA AI Enterprise, developers can build advanced AI capabilities such as computer vision, natural language processing and recommendation systems directly in their media applications. And they can prototype, test and deploy sophisticated AI models within their media workflows.Video Editors and Enthusiasts Rejoice!Holoscan for Media will be on display at IBC, running Sept. 13-16. At the Dell Technologies booth 7.A45, attendees can witness live demonstrations that showcase how to seamlessly transition from application development to live deployment.A number of NVIDIA partners will spotlight their latest RTX AI-powered video editing tools and technologies at the show.Blackmagic Designs DaVinci Resolve 19 Studio is now available, introducing AI features that streamline editing workflows:IntelliTrack AI makes it fast and easy to stabilize footage during the editing process. It can be used in DaVinci Resolves Fairlight tool to track on-screen subjects and automatically generate audio panning as they move across 2D and 3D spaces. With the AI-powered feature, editors can quickly pan or move audio across the stereo field, controlling the voice positions of multiple actors in the mix environment.UltraNR is an AI-accelerated denoise mode in DaVinci Resolves spatial noise reduction palette. Editors can use it to dramatically reduce digital noise undesired color or luminance fluctuations that obscure detail from a frame while maintaining image clarity. Editors can also combine the tool with temporal noise reduction for even more effective denoising in images with motion, where fluctuations can be more noticeable.RTX Video Super Resolution uses AI to sharpen low-resolution video. It can detect and remove compression artifacts, greatly enhancing lower-quality video.RTX Video HDR uses an AI-enhanced algorithm to remap standard dynamic range video into vibrant HDR10 color spaces. This lets video editors create high dynamic range content even if they dont have cameras capable of recording in HDR.IntelliTrack and UltraNR get a performance boost when running on NVIDIA RTX PCs and workstations. NVIDIA TensorRT lets them run up to 3x faster on a GeForce GTX 4090 laptop than a Macbook Pro M3 Max.All DaVinci Resolve AI effects are accelerated on RTX GPUs by TensorRT. The Resolve update includes GPU acceleration for its Beauty, Edge Detect and Watercolor effects, doubling their performance on NVIDIA GPUs.The update also introduces NVIDIAs H.265 Ultra-High-Quality (UHQ) mode, which utilizes NVENC to boost HEVC encoding efficiency by 10%.Pixel-Perfect Partners: Topaz Video AI and Adobe After EffectsThis year, Topaz Labs introduced an Adobe After Effects plug-in for Video AI, a leading solution for video upscaling and frame interpolation. The plug-in integrates the full range of enhancement and frame interpolation models directly into the industry-standard motion graphics software.It also allows users to access AI upscaling tools in their After Effects compositions, providing greater flexibility and faster compositing without the need to transfer large files between different tools.A standout feature of Topaz Video AI is its ability to create dramatic slow-motion videos with Topazs Apollo AI model, which can convert footage to up to 16x slow motion.Topaz Video AIs Apollo model in action slowing footage down by up to 16x using frame interpolation for breathtaking detail.The plugin also excels at upscaling, ideal for integrating low-resolution assets into larger projects without compromising quality. It includes all of Topazs enhancement models, like the Rhea model for 4x upscaling. Check out Adobes blog to learn more about After Effects plug-ins and how to use them.Built for speed, the plug-in is accelerated on RTX GPUs by NVIDIA TensorRT, boosting AI performance by up to 70%. A future update to Video AI will introduce further TensorRT performance improvements and efficiency optimizations, including a significant reduction in the number of AI model files required as part of the app installation.With the rapid integration of AI, the future of broadcasting is brighter and more innovative than ever.

Enterprises are looking for increasingly powerful compute to support their AI workloads and accelerate data processing. The efficiency gained can translate to better returns for their investments in AI training and fine-tuning, and improved user experiences for AI inference.At the Oracle CloudWorld conference today, Oracle Cloud Infrastructure (OCI) announced the first zettascale OCI Supercluster, accelerated by the NVIDIA Blackwell platform, to help enterprises train and deploy next-generation AI models using more than 100,000 of NVIDIAs latest-generation GPUs.OCI Superclusters allow customers to choose from a wide range of NVIDIA GPUs and deploy them anywhere: on premises, public cloud and sovereign cloud. Set for availability in the first half of next year, the Blackwell-based systems can scale up to 131,072 Blackwell GPUs with NVIDIA ConnectX-7 NICs for RoCEv2 or NVIDIA Quantum-2 InfiniBand networking to deliver an astounding 2.4 zettaflops of peak AI compute to the cloud. (Read the press release to learn more about OCI Superclusters.)At the show, Oracle also previewed NVIDIA GB200 NVL72 liquid-cooled bare-metal instances to help power generative AI applications. The instances are capable of large-scale training with Quantum-2 InfiniBand and real-time inference of trillion-parameter models within the expanded 72-GPU NVIDIA NVLink domain, which can act as a single, massive GPU.This year, OCI will offer NVIDIA HGX H200 connecting eight NVIDIA H200 Tensor Core GPUs in a single bare-metal instance via NVLink and NVLink Switch, and scaling to 65,536 H200 GPUs with NVIDIA ConnectX-7 NICs over RoCEv2 cluster networking. The instance is available to order for customers looking to deliver real-time inference at scale and accelerate their training workloads. (Read a blog on OCI Superclusters with NVIDIA B200, GB200 and H200 GPUs.)OCI also announced general availability of NVIDIA L40S GPU-accelerated instances for midrange AI workloads, NVIDIA Omniverse and visualization. (Read a blog on OCI Superclusters with NVIDIA L40S GPUs.)For single-node to multi-rack solutions, Oracles edge offerings provide scalable AI at the edge accelerated by NVIDIA GPUs, even in disconnected and remote locations. For example, smaller-scale deployments with Oracles Roving Edge Device v2 will now support up to three NVIDIA L4 Tensor Core GPUs.Companies are using NVIDIA-powered OCI Superclusters to drive AI innovation. Foundation model startup Reka, for example, is using the clusters to develop advanced multimodal AI models to develop enterprise agents.Rekas multimodal AI models, built with OCI and NVIDIA technology, empower next-generation enterprise agents that can read, see, hear and speak to make sense of our complex world, said Dani Yogatama, cofounder and CEO of Reka. With NVIDIA GPU-accelerated infrastructure, we can handle very large models and extensive contexts with ease, all while enabling dense and sparse training to scale efficiently at cluster levels.Accelerating Generative AI Oracle Database WorkloadsOracle Autonomous Database is gaining NVIDIA GPU support for Oracle Machine Learning notebooks to allow customers to accelerate their data processing workloads on Oracle Autonomous Database.At Oracle CloudWorld, NVIDIA and Oracle are partnering to demonstrate three capabilities that show how the NVIDIA accelerated computing platform could be used today or in the future to accelerate key components of generative AI retrieval-augmented generation pipelines.The first will showcase how NVIDIA GPUs can be used to accelerate bulk vector embeddings directly from within Oracle Autonomous Database Serverless to efficiently bring enterprise data closer to AI. These vectors can be searched using Oracle Database 23ais AI Vector Search.The second demonstration will showcase a proof-of-concept prototype that uses NVIDIA GPUs, NVIDIA RAPIDS cuVS and an Oracle-developed offload framework to accelerate vector graph index generation, which significantly reduces the time needed to build indexes for efficient vector searches.The third demonstration illustrates how NVIDIA NIM, a set of easy-to-use inference microservices, can boost generative AI performance for text generation and translation use cases across a range of model sizes and concurrency levels.Together, these new Oracle Database capabilities and demonstrations highlight how NVIDIA GPUs can be used to help enterprises bring generative AI to their structured and unstructured data housed in or managed by an Oracle Database.Sovereign AI WorldwideNVIDIA and Oracle are collaborating to deliver sovereign AI infrastructure worldwide, helping address the data residency needs of governments and enterprises.Brazil-based startup Wide Labs trained and deployed Amazonia IA, one of the first large language models for Brazilian Portuguese, using NVIDIA H100 Tensor Core GPUs and the NVIDIA NeMo framework in OCIs Brazilian data centers to help ensure data sovereignty.Developing a sovereign LLM allows us to offer clients a service that processes their data within Brazilian borders, giving Amaznia a unique market position, said Nelson Leoni, CEO of Wide Labs. Using the NVIDIA NeMo framework, we successfully trained Amaznia IA.In Japan, Nomura Research Institute, a leading global provider of consulting services and system solutions, is using OCIs Alloy infrastructure with NVIDIA GPUs to enhance its financial AI platform with LLMs operating in accordance with financial regulations and data sovereignty requirements.Communication and collaboration company Zoom will be using NVIDIA GPUs in OCIs Saudi Arabian data centers to help support compliance with local data requirements.And geospatial modeling company RSS-Hydro is demonstrating how its flood mapping platform built on the NVIDIA Omniverse platform and powered by L40S GPUs on OCI can use digital twins to simulate flood impacts in Japans Kumamoto region, helping mitigate the impact of climate change.These customers are among numerous nations and organizations building and deploying domestic AI applications powered by NVIDIA and OCI, driving economic resilience through sovereign AI infrastructure.Enterprise-Ready AI With NVIDIA and OracleEnterprises can accelerate task automation on OCI by deploying NVIDIA software such as NIM microservices and NVIDIA cuOpt with OCIs scalable cloud solutions. These solutions enable enterprises to quickly adopt generative AI and build agentic workflows for complex tasks like code generation and route optimization.NVIDIA cuOpt, NIM, RAPIDS and more are included in the NVIDIA AI Enterprise software platform, available on the Oracle Cloud Marketplace.Learn More at Oracle CloudWorldJoin NVIDIA at Oracle CloudWorld 2024 to learn how the companies collaboration is bringing AI and accelerated data processing to the worlds organizations.Register to the event to watch sessions, see demos and join Oracle and NVIDIA for the solution keynote, Unlock AI Performance with NVIDIAs Accelerated Computing Platform (SOL3866), on Wednesday, Sept. 11, in Las Vegas.

To accelerate autonomous vehicle development and deployment timelines, Nuro announced today it will license its Nuro Driver autonomous driving system directly to automakers and mobility providers.The Nuro Driver is built on NVIDIAs end-to-end safety architecture, which includes NVIDIA GPUs for AI training in the cloud and an automotive-grade NVIDIA DRIVE Thor computer running the NVIDIA DriveOS operating system inside the vehicle.The Nuro Driver has demonstrated its reliability and safety in real-world conditions with more than 1 million autonomous miles completed across its fleet of R&D vehicles and zero at-fault incidents.Its not a question of if, but when L4 autonomy will become widespread, said Jiajun Zhu, cofounder and CEO at Nuro. We believe Nuro is positioned to be a major contributor to this autonomous future where people and goods mobility are free-flowing, representing a significant increase in the quality of life for everyone.The licensing of the Nuro Driver marks a significant step forward in bringing level 4 vehicles to market, accelerating the adoption of autonomous technology across the transportation industry.An End-to-End Approach With NVIDIA DRIVENuro announced at GTC in March that the Nuro Driver, which enables level 4 autonomous driving for multiple vehicle types, is being built on NVIDIA DRIVE Thor, running on the NVIDIA DriveOS operating system for safe, AI-defined autonomous vehicles.DRIVE Thor integrates the NVIDIA Blackwell architecture, designed for transformer, large language models and generative AI workloads. Nuro also uses NVIDIA GPUs for AI training.Built with NVIDIAs end-to-end safety AV architecture, the Nuro Driver can integrate sensor processing and other safety-critical capabilities, along with AI-driven autonomy, into a single, centralized computing system, said Rishi Dhall, vice president of automotive at NVIDIA. This enables the reliability and performance needed for safe deployment of autonomous vehicles at scale.The next-gen Nuro Driver will include safety features such as microphones for siren detection and systems for removing dirt from sensors as well as redundancy in safety-critical systems.Advantages of LicensingNuros licensing model will offer automotive manufacturers and mobility companies access to a commercially independent, road-proven platform that can accelerate their autonomous vehicle development and deployment timelines.With a focus on advancing autonomy, Nuro is poised to help shape the future of transportation by driving industry-wide adoption and commercialization of autonomous technology across a broad range of vehicles and mobility applications.Test Area ExpansionNuro this summer received approval from the California Department of Motor Vehicles to test its driverless vehicles based on the Nuro Driver in four San Francisco Bay Area cities: Los Altos, Menlo Park, Mountain View and Palo Alto.The DMV permit allows Nuro vehicles to travel at any time of the day, as well as in light rain and light to moderate fog conditions.Nuro is also conducting commercial testing and delivery services in Houston.

Companies in broadcast, sports and streaming are transitioning to software-defined infrastructure to benefit from flexible deployment and to more easily adopt the latest AI technologies.NVIDIA Holoscan for Media, now in limited availability, is an AI-enabled, software-defined platform that allows live media and video pipelines to run on the same infrastructure as AI. This enables companies with live media pipelines to use applications from an ecosystem of developers on repurposable, NVIDIA-accelerated, commercial off-the-shelf hardware to enhance production and delivery.Holoscan for Media offers a unified platform for live media applications from established and emerging vendors, covering AI captioning, live graphics, vision mixing, playout server, encode, decode, transcode, multiviewer and Networked Media Open Specifications (NMOS) controller, with more being made available in coming months.Developers can use Holoscan for Media to simplify the development process, streamline delivery to customers and integrate emerging technologies all while optimizing R&D spend.Holoscan for Media is an internet protocol-based platform built on industry standards, such as ST 2110, and common application programming interfaces, meeting the strictest density and compliance requirements. Its inclusive of essential services like Precision Time Protocol, aka PTP, and NMOS for interoperability and manageability, and equipped to perform in the high-pressure production environments that comprise live broadcast.Industry Adoption of NVIDIA Holoscan for MediaCompanies with live media pipelines are embracing software-defined infrastructure as they transition to the next phase of live media production and delivery. And the ecosystem of partners who share this vision for the future of the industry, including Beamr, Harmonic, Pebble, Qvest, RAVEL, RT Software, Speechmatics and Spicy Mango continues to grow.The Holoscan for Media platform leverages the powerful integration of live video and AI. This integration, accelerated by NVIDIA computing, aligns naturally with Beamrs advanced video technology and products, said Sharon Carmel, CEO of Beamr. We are confident that our Holoscan for Media application will significantly enhance media pipelines performance by optimizing 4K p60 Live video streams with high efficiency.NVIDIA is laying the foundation for software-defined broadcast, enhancing live media with expansive compute capabilities and a developer-friendly ecosystem, said Christophe Ponsart, executive vice president and generative AI practice co-lead of Qvest, a global leader in technology and business consulting. This level of local compute, alongside NVIDIAs powerful developer tools, empowers Qvest as a technology partner and integrator to rapidly innovate, using our deep industry expertise and customer relationships to make a meaningful impact.NVIDIA Holoscan for Media, using the power of Red Hat OpenShift, delivers a scalable, cloud-native platform for next-generation live media applications, said Gino Grano, global vice president of Americas, telco, media and entertainment at Red Hat, the industry-leading Kubernetes-powered hybrid cloud platform. With this enterprise-grade open-source solution, cable and broadcast companies can benefit from more seamless deployments and management of media applications, delivering enhanced flexibility and performance across environments.Speechmatics is delighted to extend our collaboration with NVIDIA and become the first speech-to-text provider on Holoscan for Media, said David Agmen-Smith, director of product at Speechmatics, a leading provider of speech AI technology. The combination allows lightning-quick and highly accurate captions to be broadcast with incredible ease.Get StartedTake advantage of flexible deployment, resource scalability and the latest video, predictive and generative AI technologies by transitioning to true software-defined infrastructure with Holoscan for Media.Attendees of the IBC 2024 content and technology event, running Sept. 13-16 in Amsterdam, can see Holoscan for Media in action across the show floor.See notice regarding software product information.

Editors note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, software, tools and accelerations for RTX PC users.At the IFA Berlin consumer electronics and home appliances trade show this week, new RTX AI PCs will be announced, powered by RTX GPUs for advanced AI in gaming, content creation, development and academics and a neural processing unit (NPU) for offloading lightweight AI.RTX GPUs, built with specialized AI hardware called Tensor Cores, provide the compute performance needed to run the latest and most demanding AI models. They now accelerate more than 600 AI-enabled games and applications, with more than 100 million GeForce RTX and NVIDIA RTX GPUs in users hands worldwide.Since the launch of NVIDIA DLSS the first widely deployed PC AI technology more than five years ago, on-device AI has expanded beyond gaming to livestreaming, content creation, software development, productivity and STEM use cases.Accelerating AIAI boils down to massive matrix multiplication in other words, incredibly complex math. CPUs can do math, but, as serial processors, they can only perform one operation per CPU core at a time. This makes them far too slow for practical use with AI.GPUs, on the other hand, are parallel processors, performing multiple operations at once. With hundreds of Tensor Cores each and being optimized for AI, RTX GPUs can accelerate incredibly complex mathematical operations.RTX-powered systems give users a powerful GPU accelerator for demanding AI workloads in gaming, content creation, software development and STEM subjects. Some also include an NPU, a lightweight accelerator for offloading select low-power workloads.Local accelerators make AI capabilities always available (even without an internet connection), offer low latency for high responsiveness and increase privacy so that users dont have to upload sensitive materials to an online database before they become usable by an AI model.Advanced Processing PowerNVIDIA powers much of the worlds AI from data center to the edge to an install base of over 100 million PCs worldwide.The GeForce RTX and NVIDIA RTX GPUs found in laptops, desktops and workstations share the same architecture as cloud servers and provide up to 686 AI trillion operations operations per second (TOPS) across the GeForce RTX 40 Series Laptop GPU lineup.RTX GPUs unlock top-tier performance and power a wider range of AI and generative AI than systems with just an integrated system-on-a-chip (SoC).Many projects, especially within Windows, are built for and expect to run on NVIDIA cards. In addition to the wide software support base, NVIDIA GPUs also have an advantage in terms of raw performance. Jon Allman, industry analyst at Puget SystemsGamers can use DLSS for AI-enhanced performance and can look forward to NVIDIA ACE digital human technology for next-generation in-game experiences. Creators can use AI-accelerated video and photo editing tools, asset generators, AI denoisers and more. Everyday users can tap RTX Video Super Resolution and RTX Video HDR for improved video quality, and NVIDIA ChatRTX and NVIDIA Broadcast for productivity improvements. And developers can use RTX-powered coding and debugging tools, as well as the NVIDIA RTX AI Toolkit to build and deploy AI-enabled apps for RTX.Large language models like Googles Gemma, Metas Llama and Microsofts Phi all run faster on RTX AI PCs, as systems with GPUs load LLMs into VRAM. Add in NVIDIA TensorRT-LLM acceleration and RTX GPUs can run LLMs 10-100x faster than on CPUs.New RTX AI PCs Available NowNew systems from ASUS and MSI are now shipping with up to GeForce RTX 4070 Laptop GPUs delivering up to 321 AI TOPS of performance and power-efficient SoCs with Windows 11 AI PC capabilities. Windows 11 AI PCs will receive a free update to Copilot+ PC experiences when available.ASUS Zephyrus G16 comes with up to a GeForce RTX 4070 Laptop GPU to supercharge photo and video editing, image generation and coding, while game-enhancing features like DLSS create additional high-quality frames and improve image quality. The 321 TOPS of local AI processing power available from the GeForce RTX 4070 Laptop GPU enables multiple AI applications to run simultaneously, changing the way gamers, creators and engineers work and play.The ASUS ProArt P16 is the first AI PC built for advanced AI workflows across creativity, gaming, productivity and more. Its GeForce RTX 4070 Laptop GPU provides creatives with RTX AI acceleration in top 2D, 3D, video editing and streaming apps. The ASUS ProArt P13 also comes with state-of-the-art graphics and an OLED touchscreen for ease of creation. Both laptops also come NVIDIA Studio-validated, enabling and accelerating your creativity.The MSI Stealth A16 AI+ features the latest GeForce RTX 40 Series Laptop GPUs, delivering up to 321 AI TOPS with a GeForce RTX 4070 Laptop GPU. This fast and intelligent AI-powered PC is designed to excel in gaming, creation and productivity, offering access to next-level technology.These laptops join hundreds of RTX AI PCs available today from top manufacturers, with support for the 600+ AI applications and games accelerated by RTX.Generative AI is transforming graphics and interactive experiences of all kinds. Make sense of whats new and whats next by subscribing to theAI Decoded newsletter.

Edward Mehr works where AI meets the anvil. The company he cofounded, Machina Labs, blends the latest advancements in robotics and AI to form metal into countless shapes for use in defense, aerospace, and more. The companys applications accelerate design and innovation, enabling rapid iteration and production in days instead of the months required by conventional processes. NVIDIA AI Podcast host Noah Kravitz speaks with Mehr, CEO of Machina Labs, on how the company uses AI to develop the first-ever robotic blacksmith. Its Robotic Craftsman platform integrates seven-axis robots that can shape, scan, trim and drill a wide range of materials all capabilities made possible through AI.The AI Podcast Machina Labs Edward Mehr on Autonomous Blacksmith Bots and MoreTime Stamps1:12: What does Machina Labs do?3:37: Mehrs background8:45: Machina Labs manufacturing platform, the Robotic Craftsman10:39: Machina Labs history and how AI plays a role in its work15:07: The versatility of the Robotic Craftsman21:48: How the Robotic Craftsman was trained in simulations using AI-generated manufacturing data28:10: From factory to household Mehrs insight on the future of robotic applicationsYou Might Also Like:How Two Stanford Students Are Building Robots for Handling Household Chores Ep. 224BEHAVIOR-1K is a robot that can perform 1,000 household chores, including picking up fallen objects or cooking. In this episode, Stanford Ph.D. students Chengshu Eric Li and Josiah David Wong discuss the breakthroughs and challenges they experienced while developing BEHAVIOR-1K.Hittin the Sim: NVIDIAs Matt Cragun on Conditioning Autonomous Vehicles in Simulation Ep. 185NVIDIA DRIVE Sim, built on Omniverse, provides a virtual proving ground for AV testing and validation. Its a highly accurate simulation platform that can enable groundbreaking tools including synthetic data and neural reconstruction to build digital twins of driving environments. In this episode, Matt Cragun, senior product manager for AV simulation at NVIDIA, details the origins and inner workings of DRIVE Sim.NVIDIAs Liila Torabi Talks the New Era of Robotics Through Isaac Sim Ep. 147Robotics are not just limited to the assembly line. Liila Torabi, senior product manager for NVIDIA Isaac Sim, works on making the next generation of robotics possible. In this episode, she discusses the new era of robotics one driven by making robots smarter through AI.Art(ificial) Intelligence: Pindar Van Arman Builds Robots That Paint Ep. 129Pindar Van Arman is an American artist and roboticist, designing painting robots that explore the intersection of human and computational creativity. Hes built multiple artificially creative robots, the most famous of which being Cloud Painter, which was awarded first place at Robotart 2018. Tune in to hear how Van Arman deconstructs his own artistic process and teaches it to robots.Subscribe to the AI PodcastGet the AI Podcast through iTunes, Google Play, Amazon Music, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn.Make the AI Podcast better: Have a few minutes to spare? Fill out this listener survey.

Volvo Cars new, fully electric EX90 is making its way from the automakers assembly line in Charleston, South Carolina, to dealerships around the U.S.To ensure its customers benefit from future improvements and advanced safety features and capabilities, the Volvo EX90 is built on the NVIDIA DRIVE Orin system-on-a-chip (SoC), capable of more than 250 trillion operations per second (TOPS).Running NVIDIA DriveOS, the system delivers high-performance processing with automotive-grade functional safety in a package thats literally the size of a postage stamp. This core compute architecture handles all vehicle functions, ranging from enabling safety and driving assistance features to supporting the development of autonomous driving capabilities all while delivering a safe and comfortable user experience.The state-of-the-art SUV is an intelligent mobile device on wheels, equipped with the automakers most advanced sensor suite to date, including radar, lidar, cameras, ultrasonic sensors and more. NVIDIA DRIVE Orin enables real-time, redundant and advanced 360-degree surround-sensor data processing, supporting Volvo Cars unwavering commitment to safety.DRIVE Thor Computer Running DriveOS, Powering the Next Generation of Volvo CarsSetting its sights on the future, Volvo Cars also announced plans to migrate to the next-generation NVIDIA DRIVE Thor computer running DriveOS for its future fleet of cars.Before the end of the decade, Volvo Cars will move to NVIDIA DRIVE Thor, which boasts 1,000 TOPS quadrupling the processing power of a single DRIVE Orin SoC, while improving energy efficiency sevenfold.The next-generation DRIVE Thor autonomous vehicle processor incorporates the latest NVIDIA Blackwell GPU architecture, helping unlock a new realm of possibilities and capabilities both in and around the car. This advanced platform will facilitate the deployment of safe advanced driver-assistance system (ADAS) and self-driving features and pave the way for a new era of in-vehicle experiences powered by generative AI.Highlighting Volvo Cars leap to NVIDIAs next-generation processor, Volvo Cars CEO Jim Rowan noted, With NVIDIA DRIVE Thor in our future cars, our in-house developed software becomes more scalable across our product lineup, and it helps us to continue to improve the safety in our cars, deliver best-in-class customer experiences and increase our margins.Zenseact Strategic Investment in NVIDIA DGX Computers in the CloudVolvo Cars and its software subsidiary, Zenseact, are also investing in NVIDIA DGX systems for AI model training in the cloud, helping ensure that future fleets are equipped with the most advanced and well-tested AI-powered safety features.Managing the massive amount of data needed to safely train the next generation of AI-enabled vehicles demands data-center-level compute and infrastructure.NVIDIA DGX systems provide the computational performance essential for training AI models with unprecedented efficiency. Transportation companies use them to speed autonomous technology development in a cost-effective, enterprise-ready and easy-to-deploy way.Volvo Cars and Zenseacts AI training hub, based in the Nordics, will use the systems to help catalyze multiple facets of ADAS and autonomous driving software development. A key benefit is the optimization of the data annotation process a traditionally time-consuming task involving the identification and labeling of objects for classification and recognition.The cluster of DGX systems will also enable processing of the required data for safety assurance, delivering twice the performance and potentially halving time to market.The NVIDIA DGX AI supercomputer will supercharge our AI training capabilities, making this in-house AI training center one of the largest in the Nordics, said Anders Bell, chief engineering and technology officer at Volvo Cars. By leveraging NVIDIA technology and setting up the data center, we pave a quick path to high-performing AI, ultimately helping make our products safer and better.With NVIDIA technology as the AI brain inside the car and in the cloud, Volvo Cars and Zenseact can deliver safe vehicles that allow customers to drive with peace of mind, wherever the road may lead.

Fall will be here soon, so leaf it to GeForce NOW to bring the games, with 19 joining the cloud in September.Get started with the seven games available to stream this week, and a day one PC Game Pass title, Age of Mythology: Retold, from the creators of the award-winning Age of Empires franchise Worlds Edge, Forgotten Empires and Xbox Game Studios.The Open Beta for Call of Duty: Black Ops 6 runs Sept. 6-9, offering everyone a chance to experience game-changing innovations before the title officially launches on Oct. 25. Members can stream the Battle.net and Steam versions of the Open Beta instantly this week on GeForce NOW to jump right into the action.Where Myths and Heroes CollideA vast, mythical world to explore with friends? Say no moreAge of Mythology: Retold revitalizes the classic real-time strategy game by merging its beloved elements with modern visuals.Get immersed in a mythical universe, command legendary units and call upon the powers of various gods from the Atlantean, Greek, Egyptian and Norse pantheons. The single-player experience features a 50-mission campaign, including engaging battles and myth exploration in iconic locations like Troy and Midgard. Challenge friends in head-to-head matches or cooperate to take on advanced, AI-powered opponents.Call upon the gods from the cloud with an Ultimate and Priority membership and stream the game across devices. Games update automatically in the cloud, so members can dive into the action without having to wait.September Gets Better With New GamesChoose your fate.Catch the storytelling prowess of Supermassive Games in The Casting of Frank Stone, available to stream this week for members. The shadow of Frank Stone looms over Cedar Hills, a town forever altered by his violent past. Delve into the mystery of Cedar Hills alongside an original cast of characters bound together on a twisted journey where nothing is quite as it seems. Every decision shapes the story and impacts the fate of the characters.In addition, members can look for the following games this week:The Casting of Frank Stone (New release on Steam, Sept. 3)Age of Mythology (New release on Steam and Xbox, available on PC Game Pass, Sept.4 )Sniper Ghost Warrior Contracts (New release on Epic Games Store, early access Sept. 5)Warhammer 40,000: Space Marine 2 (New release on Steam, early access Sept. 5)Crime Scene Cleaner (Steam)FINAL FANTASY XVI Demo (Epic Games Store)Sins of a Solar Empire II (Steam)Heres what members can expect for the rest of September:Frostpunk 2 (New release on Steam and Xbox available on PC Game Pass, Sept. 17)FINAL FANTASY XVI (New release on Steam and Epic Games Store, Sept. 17)The Plucky Squire (New release on Steam, Sept. 17)Tiny Glade (New release on Steam, Sept. 23)Disney Epic Mickey: Rebrushed (New release on Steam, Sept. 24)Greedfall II: The Dying World (New release on Steam, Sept. 24)Mechabellum ( Steam)Blacksmith Master (New release on Steam, Sept. 26)Breachway (New release on Steam, Sept. 26)REKA (New release on Steam)Test Drive Unlimited Solar Crown (New release on Steam)Riders Republic (New release on PC Game Pass, Sept. 11). To begin playing, members need to activate access, and can refer to the help article for instructions.Additions to AugustIn addition to the 18 games announced last month, 48 more joined the GeForce NOW library:Prince of Persia: The Lost Crown (Day zero release on Steam, Aug. 8)FINAL FANTASY XVI Demo (New release on Steam, Aug. 19)Black Myth: Wukong (New release on Steam and Epic Games Store, Aug. 20)GIGANTIC: RAMPAGE EDITION (Available on Epic Games Store, free Aug. 22)Skull and Bones (New release on Steam, Aug. 22)Endzone 2 (New release on Steam, Aug. 26)Age of Mythology: Retold (Advanced access on Steam, Xbox, available on PC Game Pass, Aug. 27)Core Keeper (New release on Xbox, available on PC Game Pass, Aug. 27)Alan Wakes American Nightmare (Xbox, available on Microsoft Store)Car Manufacture (Steam)Cat Quest III (Steam)Commandos 3 HD Remaster (Xbox, available on Microsoft Store)Cooking Simulator (Xbox, available on PC Game Pass)Crown Trick (Xbox, available on Microsoft Store)Darksiders Genesis (Xbox, available on Microsoft Store)Desperados III (Xbox, available on Microsoft Store)The Dungeon of Naheulbeuk: The Amulet of Chaos (Xbox, available on Microsoft Store)Expeditions: Rome (Xbox, available on Microsoft Store)The Flame in the Flood (Xbox, available on Microsoft Store)FTL: Faster Than Light (Xbox, available on Microsoft Store)Genesis Noir (Xbox, available on PC Game Pass)House Flipper (Xbox, available on PC Game Pass)Into the Breach (Xbox, available on Microsoft Store)Iron Harvest (Xbox, available on Microsoft Store)The Knight Witch (Xbox, available on Microsoft Store)Lightyear Frontier (Xbox, available on PC Game Pass)Medieval Dynasty (Xbox, available on PC Game Pass)Metro Exodus Enhanced Edition (Xbox, available on Microsoft Store)My Time at Portia (Xbox, available on PC Game Pass)Night in the Woods (Xbox, available on Microsoft Store )Offworld Trading Company (Xbox, available on PC Game Pass)Orwell: Keeping an Eye on You (Xbox, available on Microsoft Store)Outlast 2 (Xbox, available on Microsoft Store)Project Winter (Xbox, available on Microsoft Store)Psychonauts (Steam)Psychonauts 2 (Steam and Xbox, available on PC Game Pass)Shadow Tactics: Blades of the Shogun (Xbox, available on Microsoft Store)Sid Meiers Civilization VI (Steam, Epic Games Store and Xbox, available on the Microsoft store)Sid Meiers Civilization V (Steam)Sid Meiers Civilization IV (Steam)Sid Meiers Civilization: Beyond Earth (Steam)Spirit of the North (Xbox, available on PC Game Pass)SteamWorld Heist II (Steam, Xbox, available on Microsoft Store)Visions of Mana Demo (Steam)This War of Mine (Xbox, available on PC Game Pass)We Were Here Too (Steam)Wreckfest (Xbox, available on PC Game Pass)Yokus Island Express (Xbox, available on Microsoft Store)Breachway was originally included in the August games list, but the launch date was moved to September by the developer. Stay tuned to GFN Thursday for updates.Starting in October, members will no longer see the option of launching Epic Games Store versions of games published by Ubisoft on GeForce NOW. To play these supported games, members can select the Ubisoft Connect option on GeForce NOW and will need to connect their Ubisoft Connect and Epic game store accounts the first time they play the game. Check out more details.What are you planning to play this weekend? Let us know on X or in the comments below.If you could be a god from a video game, who would you be and why? NVIDIA GeForce NOW (@NVIDIAGFN) September 4, 2024

Customer service departments across industries are facing increased call volumes, high customer service agent turnover, talent shortages and shifting customer expectations.Customers expect both self-help options and real-time, person-to-person support. These expectations for seamless, personalized experiences extend across digital communication channels, including live chat, text and social media.Despite the rise of digital channels, many consumers still prefer picking up the phone for support, placing strain on call centers. As companies strive to enhance the quality of customer interactions, operational efficiency and costs remain a significant concern.To address these challenges, businesses are deploying AI-powered customer service software to boost agent productivity, automate customer interactions and harvest insights to optimize operations.In nearly every industry, AI systems can help improve service delivery and customer satisfaction. Retailers are using conversational AI to help manage omnichannel customer requests, telecommunications providers are enhancing network troubleshooting, financial institutions are automating routine banking tasks, and healthcare facilities are expanding their capacity for patient care.What Are the Benefits of AI for Customer Service?With strategic deployment of AI, enterprises can transform customer interactions through intuitive problem-solving to build greater operational efficiencies and elevate customer satisfaction.By harnessing customer data from support interactions, documented FAQs and other enterprise resources, businesses can develop AI tools that tap into their organizations unique collective knowledge and experiences to deliver personalized service, product recommendations and proactive support.Customizable, open-source generative AI technologies such as large language models (LLMs), combined with natural language processing (NLP) and retrieval-augmented generation (RAG), are helping industries accelerate the rollout of use-case-specific customer service AI. According to McKinsey, over 80% of customer care executives are already investing in AI or planning to do so soon.With cost-efficient, customized AI solutions, businesses are automating management of help-desk support tickets, creating more effective self-service tools and supporting their customer service agents with AI assistants. This can significantly reduce operational costs and improve the customer experience.Developing Effective Customer Service AIFor satisfactory, real-time interactions, AI-powered customer service software must return accurate, fast and relevant responses. Some tricks of the trade include:Open-source foundation models can fast-track AI development. Developers can flexibly adapt and enhance these pretrained machine learning models, and enterprises can use them to launch AI projects without the high costs of building models from scratch.RAG frameworks connect foundation or general-purpose LLMs to proprietary knowledge bases and data sources, including inventory management and customer relationship management systems and customer service protocols. Integrating RAG into conversational chatbots, AI assistants and copilots tailors responses to the context of customer queries.Human-in-the-loop processes remain crucial to both AI training and live deployments. After initial training of foundation models or LLMs, human reviewers should judge the AIs responses and provide corrective feedback. This helps to guard against issues such as hallucination where the model generates false or misleading information, and other errors including toxicity or off-topic responses. This type of human involvement ensures fairness, accuracy and security is fully considered during AI development.Human participation is even more important for AI in production. When an AI is unable to adequately resolve a customer question, the program must be able to route the call to customer support teams. This collaborative approach between AI and human agents ensures that customer engagement is efficient and empathetic.Whats the ROI of Customer Service AI?The return on investment of customer service AI should be measured primarily based on efficiency gains and cost reductions. To quantify ROI, businesses can measure key indicators such as reduced response times, decreased operational costs of contact centers, improved customer satisfaction scores and revenue growth resulting from AI-enhanced services.For instance, the cost of implementing an AI chatbot using open-source models can be compared with the expenses incurred by routing customer inquiries through traditional call centers. Establishing this baseline helps assess the financial impact of AI deployments on customer service operations.To solidify understanding of ROI before scaling AI deployments, companies can consider a pilot period. For example, by redirecting 20% of call center traffic to AI solutions for one or two quarters and closely monitoring the outcomes, businesses can obtain concrete data on performance improvements and cost savings. This approach helps prove ROI and informs decisions for further investment.Businesses across industries are using AI for customer service and measuring their success:Retailers Reduce Call Center LoadModern shoppers expect smooth, personalized and efficient shopping experiences, whether in store or on an e-commerce site. Customers of all generations continue prioritizing live human support, while also desiring the option to use different channels. But complex customer issues coming from a diverse customer base can make it difficult for support agents to quickly comprehend and resolve incoming requests.To address these challenges, many retailers are turning to conversational AI and AI-based call routing. According to NVIDIAs 2024 State of AI in Retail and CPG report, nearly 70% of retailers believe that AI has already boosted their annual revenue.CP All, Thailands sole licensed operator for 7-Eleven convenience stores, has implemented conversational AI chatbots in its call centers, which rack up more than 250,000 calls per day. Training the bots presented unique challenges due to the complexities of the Thai language, which includes 21 consonants, 18 pure vowels, three diphthongs and five tones.To manage this, CP All used NVIDIA NeMo, a framework designed for building, training and fine-tuning GPU-accelerated speech and natural language understanding models. With automatic speech recognition and NLP models powered by NVIDIA technologies, CP Alls chatbot achieved a 97% accuracy rate in understanding spoken Thai.With the conversational chatbot handling a significant number of customer conversations, the call load on human agents was reduced by 60%. This allowed customer service teams to focus on more complex tasks. The chatbot also helped reduce wait times and provided quicker, more accurate responses, leading to higher customer satisfaction levels.With AI-powered support experiences, retailers can enhance customer retention, strengthen brand loyalty and boost sales.Telecommunications Providers Automate Network TroubleshootingTelecommunications providers are challenged to address complex network issues while adhering to service-level agreements with end customers for network uptime. Maintaining network performance requires rapid troubleshooting of network devices, pinpointing root causes and resolving difficulties at network operations centers.With its abilities to analyze vast amounts of data, troubleshoot network problems autonomously and execute numerous tasks simultaneously, generative AI is ideal for network operations centers. According to an IDC survey, 73% of global telcos have prioritized AI and machine learning investments for operational support as their top transformation initiative, underscoring the industrys shift toward AI and advanced technologies.Infosys, a leader in next-generation digital services and consulting, has built AI-driven solutions to help its telco partners overcome customer service challenges. Using NVIDIA NIM microservices and RAG, Infosys developed an AI chatbot to support network troubleshooting.By offering quick access to essential, vendor-agnostic router commands for diagnostics and monitoring, the generative AI-powered chatbot significantly reduces network resolution times, enhancing overall customer support experiences.To ensure accuracy and contextual responses, Infosys trained the generative AI solution on telecom device-specific manuals, training documents and troubleshooting guides. Using NVIDIA NeMo Retriever to query enterprise data, Infosys achieved 90% accuracy for its LLM output. By fine-tuning and deploying models with NVIDIA technologies, Infosys achieved a latency of 0.9 seconds, a 61% reduction compared with its baseline model. The RAG-enabled chatbot powered by NeMo Retriever also attained 92% accuracy, compared with the baseline models 85%.With AI tools supporting network administrators, IT teams and customer service agents, telecom providers can more efficiently identify and resolve network issues.Financial Services Institutions Pinpoint Fraud With EaseWhile customers expect anytime, anywhere banking and support, financial services require a heightened level of data sensitivity. And unlike other industries that may include one-off purchases, banking is typically based on ongoing transactions and long-term customer relationships.At the same time, user loyalty can be fleeting, with up to 80% of banking customers willing to switch institutions for a better experience. Financial institutions must continuously improve their support experiences and update their analyses of customer needs and preferences.Many banks are turning to AI virtual assistants that can interact directly with customers to manage inquiries, execute transactions and escalate complex issues to human customer support agents. According to NVIDIAs 2024 State of AI in Financial Services report, more than one-fourth of survey respondents are using AI to enhance customer experiences, and 34% are exploring the use of generative AI and LLMs for customer experience and engagement.Bunq, a European digital bank with more than 2 million customers and 8 billion euros worth of deposits, is deploying generative AI to meet user needs. With proprietary LLMs, the company built Finn, a personal AI assistant available to both customers and bank employees. Finn can answer finance-related inquiries such as How much did I spend on groceries last month? or What is the name of the Indian restaurant I ate at last week?Plus, with a human-in-the-loop process, Finn helps employees more quickly identify fraud. By collecting and analyzing data for compliance officers to review, bunq now identifies fraud in just three to seven minutes, down from 30 minutes without Finn.By deploying AI tools that can use data to protect customer transactions, execute banking requests and act on customer feedback, financial institutions can serve customers at a higher level, building the trust and satisfaction necessary for long-term relationships.Healthcare and Life Sciences Organizations Overcome Staffing ShortagesIn healthcare, patients need quick access to medical expertise, precise and tailored treatment options, and empathetic interactions with healthcare professionals. But with the World Health Organization estimating a 10 million personnel shortage by 2030, access to quality care could be jeopardized.AI-powered digital healthcare assistants are helping medical institutions do more with less. With LLMs trained on specialized medical corpuses, AI copilots can save physicians and nurses hours of daily work by helping with clinical note-taking, automating order-placing for prescriptions and lab tests, and following up with after-visit patient notes.Multimodal AI that combines language and vision models can make healthcare settings safer by extracting insights and providing summaries of image data for patient monitoring. For example, such technology can alert staff of patient fall risks and other patient room hazards.To support healthcare professionals, Hippocratic AI has trained a generative AI healthcare agent to perform low-risk, non-diagnostic routine tasks, like reminding patients of necessary appointment prep and following up after visits to make sure medication routines are being followed and no adverse side effects are being experienced.Hippocratic AI trained its models on evidence-based medicine and completed rigorous testing with a large group of certified nurses and doctors. The constellation architecture of the solution comprises 20 models, one of which communicates with patients while the other 19 supervise its output. The complete system contains 1.7 trillion parameters.The possibility of every doctor and patient having their own AI-powered digital healthcare assistant means reduced clinician burnout and higher-quality medical care.Raising the Bar for Customer Experiences With AIBy integrating AI into customer service interactions, businesses can offer more personalized, efficient and prompt service, setting new standards for omnichannel support experiences across platforms. With AI virtual assistants that process vast amounts of data in seconds, enterprises can equip their support agents to deliver tailored responses to the complex needs of a diverse customer base.To develop and deploy effective customer service AI, businesses can fine-tune AI models and deploy RAG solutions to meet diverse and specific needs.NVIDIA offers a suite of tools and technologies to help enterprises get started with customer service AI.NVIDIA NIM microservices, part of the NVIDIA AI Enterprise software platform, accelerate generative AI deployment and support various optimized AI models for seamless, scalable inference. NVIDIA NIM Agent Blueprints provide developers with packaged reference examples to build innovative solutions for customer service applications.By taking advantage of AI development tools, enterprises can build accurate and high-speed AI applications to transform employee and customer experiences.Learn more about improving customer service with generative AI.

The business transformations that generative AI brings come with risks that AI itself can help secure in a kind of flywheel of progress.Companies who were quick to embrace the open internet more than 20 years ago were among the first to reap its benefits and become proficient in modern network security.Enterprise AI is following a similar pattern today. Organizations pursuing its advances especially with powerful generative AI capabilities are applying those learnings to enhance their security.For those just getting started on this journey, here are ways to address with AI three of the top security threats industry experts have identified for large language models (LLMs).AI Guardrails Prevent Prompt InjectionsGenerative AI services are subject to attacks from malicious prompts designed to disrupt the LLM behind it or gain access to its data. As the report cited above notes, Direct injections overwrite system prompts, while indirect ones manipulate inputs from external sources.The best antidote for prompt injections are AI guardrails, built into or placed around LLMs. Like the metal safety barriers and concrete curbs on the road, AI guardrails keep LLM applications on track and on topic.The industry has delivered and continues to work on solutions in this area. For example, NVIDIA NeMo Guardrails software lets developers protect the trustworthiness, safety and security of generative AI services.AI Detects and Protects Sensitive DataThe responses LLMs give to prompts can on occasion reveal sensitive information. With multifactor authentication and other best practices, credentials are becoming increasingly complex, widening the scope of whats considered sensitive data.To guard against disclosures, all sensitive information should be carefully removed or obscured from AI training data. Given the size of datasets used in training, its hard for humans but easy for AI models to ensure a data sanitation process is effective.An AI model trained to detect and obfuscate sensitive information can help safeguard against revealing anything confidential that was inadvertently left in an LLMs training data.Using NVIDIA Morpheus, an AI framework for building cybersecurity applications, enterprises can create AI models and accelerated pipelines that find and protect sensitive information on their networks. Morpheus lets AI do what no human using traditional rule-based analytics can: track and analyze the massive data flows on an entire corporate network.AI Can Help Reinforce Access ControlFinally, hackers may try to use LLMs to get access control over an organizations assets. So, businesses need to prevent their generative AI services from exceeding their level of authority.The best defense against this risk is using the best practices of security-by-design. Specifically, grant an LLM the least privileges and continuously evaluate those permissions, so it can only access the tools and data it needs to perform its intended functions. This simple, standard approach is probably all most users need in this case.However, AI can also assist in providing access controls for LLMs. A separate inline model can be trained to detect privilege escalation by evaluating an LLMs outputs.Start the Journey to Cybersecurity AINo one technique is a silver bullet; security continues to be about evolving measures and countermeasures. Those who do best on that journey make use of the latest tools and technologies.To secure AI, organizations need to be familiar with it, and the best way to do that is by deploying it in meaningful use cases. NVIDIA and its partners can help with full-stack solutions in AI, cybersecurity and cybersecurity AI.Looking ahead, AI and cybersecurity will be tightly linked in a kind of virtuous cycle, a flywheel of progress where each makes the other better. Ultimately, users will come to trust it as just another form of automation.Learn more about NVIDIAs cybersecurity AI platform and how its being put to use. And listen to cybersecurity talks from experts at the NVIDIA AI Summit in October.

It all started at Berlins Merantix venture studio in 2022, when Silviu Homoceanu and Max Fischer agreed AI could play a big role in improving manufacturing. So the two started Deltia.ai, which runs NVIDIA Metropolis vision AI on NVIDIA Jetson AGX Orin modules to measure and help optimize assembly line processes.Hailing from AI backgrounds, Homoceanu had previously led self-driving software at Volkswagen, while Fischer had founded a startup that helped digitize more than 40 factories.Deltia, an NVIDIA Metropolis partner, estimates that today its software platform can provide as much as a 20% performance jump on production lines for its customers.Customers using the Deltia platform include Viessman, a maker of heating pumps, and industrial electronics company ABB, among others. Viessman is running Deltia at 15 stations, and plans to add it to even more lines in the future. Once all lines are linked to Deltia, production managers say that they expect up to a 50% increase in overall productivity.We provide our users with a dashboard that is basically the Google Analytics of manufacturing, said Homoceanu, Deltias CTO. We install these sensors, and two weeks later they get the keys to this dashboard, and the magic happens in the background.Capturing Assembly Line Insights for Digital TransformationsOnce the cameras start gathering data on assembly lines, Deltia uses that information to train models on NVIDIA-accelerated computing that can monitor activities on the line. It then uses those models deployed on Jetson AGX Orin modules at the edge to gather operational insights.These Jetson-based systems continuously monitor the camera streams and extract metadata. This metadata identifies the exact points in time when a product arrives at a specific station, when it is being worked on and when it leaves the station. This digital information is available to line managers and process improvement personnel via Deltias custom dashboard, helping to identify bottlenecks and accelerate line output.TensorRT helps us compress complex AI models to a level where we can serve, in an economical fashion, multiple stations with a single Jetson device, said Homoceanu.Tapping Into Jetson Orin for Edge AI-Based Customer InsightsBeyond identifying quick optimizations, Deltias analytics help visualize production flows hour-by-hour. This means that Deltia can send rapid alerts when production slips away from predicted target ranges, and it can continuously track output, cycle times and other critical key performance indicators.It also helps map how processes flow throughout a factory floor, and it suggests improvements for things like walking routes and shop-floor layouts. One of Deltias customers used the platform to identify that materials shelves were too far from workers, which caused unnecessarily long cycle times and limited production. Once the shelves were moved, production went up more than 30%.Deltias applications extend beyond process improvements. The platform can be used to help monitor machine states at a granular level, assisting to predict when machine parts are worn out and recommend preemptive replacements, saving time and money down the line. The platform can also suggest optimizations for energy usage, saving on operational costs and reducing maintenance expenses.Our vision is to empower manufacturers with the tools to achieve unprecedented efficiency, said Fischer, CEO of Deltia.ai. Seeing our customers experience as much as a 30% increase in productivity with our vision models running on NVIDIA Jetson Orin validates the transformative potential of our technology.Deltia is a member of the NVIDIA Inceptionprogram for cutting-edge startups.Learn more about NVIDIA Metropolis and NVIDIA Jetson.

Well before OpenAI upended the technology industry with its release of ChatGPT in the fall of 2022, Douwe Kiela already understood why large language models, on their own, could only offer partial solutions for key enterprise use cases.The young Dutch CEO of Contextual AI had been deeply influenced by two seminal papers from Google and OpenAI, which together outlined the recipe for creating fast, efficient transformer-based generative AI models and LLMs.Soon after those papers were published in 2017 and 2018, Kiela and his team of AI researchers at Facebook, where he worked at that time, realized LLMs would face profound data freshness issues.They knew that when foundation models like LLMs were trained on massive datasets, the training not only imbued the model with a metaphorical brain for reasoning across data. The training data also represented the entirety of a models knowledge that it could draw on to generate answers to users questions.Kielas team realized that, unless an LLM could access relevant real-time data in an efficient, cost-effective way, even the smartest LLM wouldnt be very useful for many enterprises needs.So, in the spring of 2020, Kiela and his team published a seminal paper of their own, which introduced the world to retrieval-augmented generation. RAG, as its commonly called, is a method for continuously and cost-effectively updating foundation models with new, relevant information, including from a users own files and from the internet. With RAG, an LLMs knowledge is no longer confined to its training data, which makes models far more accurate, impactful and relevant to enterprise users.Today, Kiela and Amanpreet Singh, a former colleague at Facebook, are the CEO and CTO of Contextual AI, a Silicon Valley-based startup, which recently closed an $80 million Series A round, which included NVIDIAs investment arm, NVentures. Contextual AI is also a member of NVIDIA Inception, a program designed to nurture startups. With roughly 50 employees, the company says it plans to double in size by the end of the year.The platform Contextual AI offers is called RAG 2.0. In many ways, its an advanced, productized version of the RAG architecture Kiela and Singh first described in their 2020 paper.RAG 2.0 can achieve roughly 10x better parameter accuracy and performance over competing offerings, Kiela says.That means, for example, that a 70-billion-parameter model that would typically require significant compute resources could instead run on far smaller infrastructure, one built to handle only 7 billion parameters without sacrificing accuracy. This type of optimization opens up edge use cases with smaller computers that can perform at significantly higher-than-expected levels.When ChatGPT happened, we saw this enormous frustration where everybody recognized the potential of LLMs, but also realized the technology wasnt quite there yet, explained Kiela. We knew that RAG was the solution to many of the problems. And we also knew that we could do much better than what we outlined in the original RAG paper in 2020.Integrated Retrievers and Language Models Offer Big Performance GainsThe key to Contextual AIs solutions is its close integration of its retriever architecture, the R in RAG, with an LLMs architecture, which is the generator, or G, in the term. The way RAG works is that a retriever interprets a users query, checks various sources to identify relevant documents or data and then brings that information back to an LLM, which reasons across this new information to generate a response.Since around 2020, RAG has become the dominant approach for enterprises that deploy LLM-powered chatbots. As a result, a vibrant ecosystem of RAG-focused startups has formed.One of the ways Contextual AI differentiates itself from competitors is by how it refines and improves its retrievers through back propagation, a process of adjusting algorithms the weights and biases underlying its neural network architecture.And, instead of training and adjusting two distinct neural networks, that is, the retriever and the LLM, Contextual AI offers a unified state-of-the-art platform, which aligns the retriever and language model, and then tunes them both through back propagation.Synchronizing and adjusting weights and biases across distinct neural networks is difficult, but the result, Kiela says, leads to tremendous gains in precision, response quality and optimization. And because the retriever and generator are so closely aligned, the responses they create are grounded in common data, which means their answers are far less likely than other RAG architectures to include made up or hallucinated data, which a model might offer when it doesnt know an answer.Our approach is technically very challenging, but it leads to much stronger coupling between the retriever and the generator, which makes our system far more accurate and much more efficient, said Kiela.Tackling Difficult Use Cases With State-of-the-Art InnovationsRAG 2.0 is essentially LLM-agnostic, which means it works across different open-source language models, like Mistral or Llama, and can accommodate customers model preferences. The startups retrievers were developed using NVIDIAs Megatron LM on a mix of NVIDIA H100 and A100 Tensor Core GPUs hosted in Google Cloud.One of the significant challenges every RAG solution faces is how to identify the most relevant information to answer a users query when that information may be stored in a variety of formats, such as text, video or PDF.Contextual AI overcomes this challenge through a mixture of retrievers approach, which aligns different retrievers sub-specialties with the different formats data is stored in.Contextual AI deploys a combination of RAG types, plus a neural reranking algorithm, to identify information stored in different formats which, together, are optimally responsive to the users query.For example, if some information relevant to a query is stored in a video file format, then one of the RAGs deployed to identify relevant data would likely be a Graph RAG, which is very good at understanding temporal relationships in unstructured data like video. If other data were stored in a text or PDF format, then a vector-based RAG would simultaneously be deployed.The neural reranker would then help organize the retrieved data and the prioritized information would then be fed to the LLM to generate an answer to the initial query.To maximize performance, we almost never use a single retrieval approach its usually a hybrid because they have different and complementary strengths, Kiela said. The exact right mixture depends on the use case, the underlying data and the users query.By essentially fusing the RAG and LLM architectures, and offering many routes for finding relevant information, Contextual AI offers customers significantly improved performance. In addition to greater accuracy, its offering lowers latency thanks to fewer API calls between the RAGs and LLMs neural networks.Because of its highly optimized architecture and lower compute demands, RAG 2.0 can run in the cloud, on premises or fully disconnected. And that makes it relevant to a wide array of industries, from fintech and manufacturing to medical devices and robotics.The use cases were focusing on are the really hard ones, Kiela said. Beyond reading a transcript, answering basic questions or summarization, were focused on the very high-value, knowledge-intensive roles that will save companies a lot of money or make them much more productive.

Its time to mana-fest the spirit of adventure with Square Enixs highly anticipated action role-playing game, Visions of Mana, launching today in the cloud.Members can also head to a galaxy far, far away, from the comfort of their homes, with the power of the cloud and Ubisofts Star Wars Outlaws, with early access available on GeForce NOW.Plus, be among the first to get early access to the Call of Duty: Black Ops 6 Open Beta on GeForce NOW without having to wait around for downloads early access runs Aug. 30-Sept. 4 for those who preorder the game. Call of Duty: Black Ops 6 will join the cloud when the full game is released Oct. 25.These triple-A titles are part of 26 titles joining the GeForce NOW library of over 2,000 games this week.Cloudy With a Chance of ManaVisions of Mana details a whimsical journey through the enchanting world of Mana. Step into the shoes of Val, a curious young man on an epic quest to escort his childhood friend Hina to the sacred Tree of Mana. Along the way, encounter a colorful cast of characters and face off against endearing yet formidable enemies.Get ready for mana madness in the cloud.The games combat system blends action and strategy for real-time battles where party members can be switched on the fly. Different party members offer unique skills, such as Vals powerful sword strikes or magical spells cast by Careenas dragon companion Ramcoh. Plus, traverse the games expansive, semi-open world on Pikuls adorable, rideable creatures that can also ram through enemies.Stream the game on an Ultimate or Priority membership for a seamless magical adventure. Experience the lush landscapes and dynamic combat in stunning detail at up to 4K resolution and 120 frames per second for Ultimate members. GeForce NOW makes it easy to stay connected to the world of Mana whether at home or on the go, ensuring the journey to the Tree of Mana is always within reach.Join the Galaxys Most WantedMake your own destiny in the cloud.In Star Wars Outlaws the highly anticipated single-player action-adventure game from Ubisoft explore the depths of the galaxys underworld as part of the beloved Star Wars franchises first-ever open-world game, set between the events of The Empire Strikes Back and Return of the Jedi.Step into the shoes of Kay Vess, a daring scoundrel seeking freedom and adventure. Navigate distinct locations across the galaxy both iconic and new and encounter bustling cities, cantinas and sprawling outdoor landscapes. Fight, steal and outwit others while joining the ranks of the galaxys most wanted. Plus, play alongside Nix, a loyal companion who helps turn any situation to Kays advantage through blaster combat, stealth and clever distractions.Get ready for high-stakes missions, space dogfights and an ever-changing reputation based on player choices. Unlock the power of the Force with a GeForce NOW Ultimate membership and stream from GeForce RTX 4080 SuperPODs at up to 4K resolution at 120 fps, without the need for upgraded hardware. The AI-powered graphics of NVIDIA DLSS 3.5 with Ray Reconstruction enhance the game for maximum performance, offering the clarity of a Jedis vision, and NVIDIA Reflex technology enables unbeatable responsiveness.Answering the CallGet ready for the anticipated addition to the Call of Duty franchise Call of Duty: Black Ops 6. GeForce NOW will support the titles PC Game Pass, Battle.net and Steam versions in the cloud.Explore a range of new features, including innovative mechanics such as Omnimovement, which allows players to sprint, slide and dive in any direction to enhance the fluidity of combat. The Black Ops 6 Campaign features a spy action thriller set in the early 90s, a period of transition and upheaval in global politics, characterized by the end of the Cold War and the rise of the United States. With a mind-bending narrative and unbound by the rules of engagement, the title embodies signature Black Ops.Experience the new gameplay mechanics and features before the titles official launch with early access to the preorder beta from Aug. 30-Sept. 4 available for those who preorder the game or have an active PC Game Pass subscription. The Open Beta follows soon after on Sept. 6-9 and will be available for all gamers to hop into the action, even without preordering the game.GeForce NOW Ultimate members can gain an advantage on the field with ultra low-latency gaming, streaming from GeForce RTX 4080 SuperPODs in the cloud.WOW, New GamesJoin the underground party from the cloud.The newest World of Warcraft expansion, The War Within, is available to play from the cloud today. Head to the subterranean realm of Khaz Algar, featuring four new zones, including the Isle of Dorn home to the Earthen, a newly playable allied race. Experience game additions such as Delves, bite-sized world instances for solo or small-group play, and Warbands, which allow players to manage and share achievements across multiple characters.In addition, GeForce NOW recently added support for over 25 of the top AddOns from CurseForge, a leading platform for WoW customization, enabling members to explore new adventures under the surface from the cloud.In addition, members can look for the following:Endzone 2 (New release on Steam, Aug. 26)Age of Mythology: Retold (New release on Steam, Xbox, available on PC Game Pass, Advanced Access on Aug. 27)Core Keeper (New release on Xbox, available on PC Game Pass, Aug. 27)Star Wars Outlaws (New release on Ubisoft Connect, early access Aug. 27)Akimbot (New release on Steam, Aug. 29)Gori: Cuddly Carnage (New release on Steam, Aug. 29)MEMORIAPOLIS (New release on Steam, Aug. 29)Visions of Mana (New release on Steam, Aug. 29)Avatar: Frontiers of Pandora (Steam)Cat Quest III (Steam)Cooking Simulator (Xbox, available on PC Game Pass)Crown Trick (Xbox, available on Microsoft Store)Darksiders Genesis (Xbox, available on Microsoft Store)Expeditions: Rome (Xbox, available on Microsoft Store)Heading Out (Steam)Into the Breach (Xbox, available on Microsoft Store)Iron Harvest (Xbox, available on Microsoft Store)The Knight Witch (Xbox, available on Microsoft Store)Lightyear Frontier (Xbox, available on PC Game Pass)Metro Exodus Enhanced Edition (Xbox, available on Microsoft Store)Outlast 2 (Xbox, available on Microsoft Store)Saturnalia (Steam)SteamWorld Heist II (Steam, Xbox, available on Microsoft Store)This War of Mine (Xbox, available on PC Game Pass)We Were Here Too (Steam)Yokus Island Express (Xbox, available on Microsoft Store)What are you planning to play this weekend? Let us know on X or in the comments below.Ready for a new adventure? What sort of magical quest would you want to go on? NVIDIA GeForce NOW (@NVIDIAGFN) August 28, 2024

As enterprises race to adopt generative AI and bring new services to market, the demands on data center infrastructure have never been greater. Training large language models is one challenge, but delivering LLM-powered real-time services is another.In the latest round of MLPerf industry benchmarks, Inference v4.1, NVIDIA platforms delivered leading performance across all data center tests. The first-ever submission of the upcoming NVIDIA Blackwell platform revealed up to 4x more performance than the NVIDIA H100 Tensor Core GPU on MLPerfs biggest LLM workload, Llama 2 70B, thanks to its use of a second-generation Transformer Engine and FP4 Tensor Cores.The NVIDIA H200 Tensor Core GPU delivered outstanding results on every benchmark in the data center category including the latest addition to the benchmark, the Mixtral 8x7B mixture of experts (MoE) LLM, which features a total of 46.7 billion parameters, with 12.9 billion parameters active per token.MoE models have gained popularity as a way to bring more versatility to LLM deployments, as theyre capable of answering a wide variety of questions and performing more diverse tasks in a single deployment. Theyre also more efficient since they only activate a few experts per inference meaning they deliver results much faster than dense models of a similar size.The continued growth of LLMs is driving the need for more compute to process inference requests. To meet real-time latency requirements for serving todays LLMs, and to do so for as many users as possible, multi-GPU compute is a must. NVIDIA NVLink and NVSwitch provide high-bandwidth communication between GPUs based on the NVIDIA Hopper architecture and provide significant benefits for real-time, cost-effective large model inference. The Blackwell platform will further extend NVLink Switchs capabilities with larger NVLink domains with 72 GPUs.In addition to the NVIDIA submissions, 10 NVIDIA partners ASUSTek, Cisco, Dell Technologies, Fujitsu, Giga Computing, Hewlett Packard Enterprise (HPE), Juniper Networks, Lenovo, Quanta Cloud Technology and Supermicro all made solid MLPerf Inference submissions, underscoring the wide availability of NVIDIA platforms.Relentless Software InnovationNVIDIA platforms undergo continuous software development, racking up performance and feature improvements on a monthly basis.In the latest inference round, NVIDIA offerings, including the NVIDIA Hopper architecture, NVIDIA Jetson platform and NVIDIA Triton Inference Server, saw leaps and bounds in performance gains.The NVIDIA H200 GPU delivered up to 27% more generative AI inference performance over the previous round, underscoring the added value customers get over time from their investment in the NVIDIA platform.Triton Inference Server, part of the NVIDIA AI platform and available with NVIDIA AI Enterprise software, is a fully featured open-source inference server that helps organizations consolidate framework-specific inference servers into a single, unified platform. This helps lower the total cost of ownership of serving AI models in production and cuts model deployment times from months to minutes.In this round of MLPerf, Triton Inference Server delivered near-equal performance to NVIDIAs bare-metal submissions, showing that organizations no longer have to choose between using a feature-rich production-grade AI inference server and achieving peak throughput performance.Going to the EdgeDeployed at the edge, generative AI models can transform sensor data, such as images and videos, into real-time, actionable insights with strong contextual awareness. The NVIDIA Jetson platform for edge AI and robotics is uniquely capable of running any kind of model locally, including LLMs, vision transformers and Stable Diffusion.In this round of MLPerf benchmarks, NVIDIA Jetson AGX Orin system-on-modules achieved more than a 6.2x throughput improvement and 2.4x latency improvement over the previous round on the GPT-J LLM workload. Rather than developing for a specific use case, developers can now use this general-purpose 6-billion-parameter model to seamlessly interface with human language, transforming generative AI at the edge.Performance Leadership All AroundThis round of MLPerf Inference showed the versatility and leading performance of NVIDIA platforms extending from the data center to the edge on all of the benchmarks workloads, supercharging the most innovative AI-powered applications and services. To learn more about these results, see our technical blog.H200 GPU-powered systems are available today from CoreWeave the first cloud service provider to announce general availability and server makers ASUS, Dell Technologies, HPE, QTC and Supermicro.See notice regarding software product information.

Editors note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, software, tools and accelerations for RTX PC users.Large language models are driving some of the most exciting developments in AI with their ability to quickly understand, summarize and generate text-based content.These capabilities power a variety of use cases, including productivity tools, digital assistants, non-playable characters in video games and more. But theyre not a one-size-fits-all solution, and developers often must fine-tune LLMs to fit the needs of their applications.The NVIDIA RTX AI Toolkit makes it easy to fine-tune and deploy AI models on RTX AI PCs and workstations through a technique called low-rank adaptation, or LoRA. A new update, available today, enables support for using multiple LoRA adapters simultaneously within the NVIDIA TensorRT-LLM AI acceleration library, improving the performance of fine-tuned models by up to 6x.Fine-Tuned for PerformanceLLMs must be carefully customized to achieve higher performance and meet growing user demands.These foundational models are trained on huge amounts of data but often lack the context needed for a developers specific use case. For example, a generic LLM can generate video game dialogue, but it will likely miss the nuance and subtlety needed to write in the style of a woodland elf with a dark past and a barely concealed disdain for authority.To achieve more tailored outputs, developers can fine-tune the model with information related to the apps use case.Take the example of developing an app to generate in-game dialogue using an LLM. The process of fine-tuning starts with using the weights of a pretrained model, such as information on what a character may say in the game. To get the dialogue in the right style, a developer can tune the model on a smaller dataset of examples, such as dialogue written in a more spooky or villainous tone.In some cases, developers may want to run all of these different fine-tuning processes simultaneously. For example, they may want to generate marketing copy written in different voices for various content channels. At the same time, they may want to summarize a document and make stylistic suggestions as well as draft a video game scene description and imagery prompt for a text-to-image generator.Its not practical to run multiple models simultaneously, as they wont all fit in GPU memory at the same time. Even if they did, their inference time would be impacted by memory bandwidth how fast data can be read from memory into GPUs.Lo(RA) and BeholdA popular way to address these issues is to use fine-tuning techniques such as low-rank adaptation. A simple way of thinking of it is as a patch file containing the customizations from the fine-tuning process.Once trained, customized LoRA adapters can integrate seamlessly with the foundation model during inference, adding minimal overhead. Developers can attach the adapters to a single model to serve multiple use cases. This keeps the memory footprint low while still providing the additional details needed for each specific use case.Architecture overview of supporting multiple clients and use-cases with a single foundation model using multi-LoRA capabilitiesIn practice, this means that an app can keep just one copy of the base model in memory, alongside many customizations using multiple LoRA adapters.This process is called multi-LoRA serving. When multiple calls are made to the model, the GPU can process all of the calls in parallel, maximizing the use of its Tensor Cores and minimizing the demands of memory and bandwidth so developers can efficiently use AI models in their workflows. Fine-tuned models using multi-LoRA adapters perform up to 6x faster.LLM inference performance on GeForce RTX 4090 Desktop GPU for Llama 3B int4 with LoRA adapters applied at runtime. Input sequence length is 1,000 tokens and output sequence length is 100 tokens. LoRA adapter max rank is 64.In the example of the in-game dialogue application described earlier, the apps scope could be expanded, using multi-LoRA serving, to generate both story elements and illustrations driven by a single prompt.The user could input a basic story idea, and the LLM would flesh out the concept, expanding on the idea to provide a detailed foundation. The application could then use the same model, enhanced with two distinct LoRA adapters, to refine the story and generate corresponding imagery. One LoRA adapter generates a Stable Diffusion prompt to create visuals using a locally deployed Stable Diffusion XL model. Meanwhile, the other LoRA adapter, fine-tuned for story writing, could craft a well-structured and engaging narrative.In this case, the same model is used for both inference passes, ensuring that the space required for the process doesnt significantly increase. The second pass, which involves both text and image generation, is performed using batched inference, making the process exceptionally fast and efficient on NVIDIA GPUs. This allows users to rapidly iterate through different versions of their stories, refining the narrative and the illustrations with ease.This process is outlined in more detail in a recent technical blog.LLMs are becoming one of the most important components of modern AI. As adoption and integration grows, demand for powerful, fast LLMs with application-specific customizations will only increase. The multi-LoRA support added today to the RTX AI Toolkit gives developers a powerful new way to accelerate these capabilities.

The initial wave of generative AI was driven by its use in internet services that showed incredible new possibilities with tools that could help people write, research and imagine faster than ever.The second wave of generative AI is now here, powered by the availability of advanced open-source foundation models, as well as advancements in agentic AI that are improving efficiency and autonomy of AI workflows. Enterprises across industries can use models like Google Gemma, Llama 3.1 405B, Microsoft Phi, Mixtral and Nemotron to develop their own AI applications to support business growth and boost productivity.To accelerate business transformation, enterprises need blueprints for canonical generative AI workflows like digital human customer service chatbots, retrieval-augmented generation and drug discovery. While NVIDIA NIM microservices help make these models efficient and accessible for enterprise use, building enterprise generative AI applications is a complex, multistep process.Launched today, NVIDIA NIM Agent Blueprints include everything an enterprise developer needs to build and deploy customized generative AI applications that make a transformative impact on business objectives.Blueprints for Data-Driven Enterprise FlywheelsNIM Agent Blueprints are reference AI workflows tailored for specific use cases. They include sample applications built with NVIDIA NIM and partner microservices, reference code, customization documentation and a Helm chart for deployment.With NIM Agent Blueprints, developers can gain a head start on creating their own applications using NVIDIAs advanced AI tools and end-to-end development experience for each use case. The blueprints are designed to be modified and enhanced, and allow developers to leverage both information retrieval and agent-based workflows capable of performing complex tasks.NIM Agent Blueprints also help developers improve their applications throughout the AI lifecycle. As users interact with AI applications, new data is generated. This data can be used to refine and enhance the models in a continuous learning cycle, creating a data-driven generative AI flywheel.NIM Agent Blueprints help enterprises build their own generative AI flywheels with applications that link models with their data. NVIDIA NeMo facilitates this process, while NVIDIA AI Foundry serves as the production environment for running the flywheel.The first NIM Agent Blueprints available are:digital human for customer servicegenerative virtual screening for accelerated drug discoverymultimodal PDF data extraction for enterprise RAGServiceNow is a leader in enterprise AI that has integrated advanced generative AI capabilities into its digital workflow platform. Its already bringing NIM microservices into its Now Assist AI solutions and working with the technologies featured in the digital human for customer service NIM Blueprint.AI is not just a tool, its the foundation of a fundamental shift in how companies can better equip employees and serve customers, said Jon Sigler, senior vice president, Platform and AI at ServiceNow. Through our collaboration with NVIDIA, weve built new generative AI products and features that are driving growth and powering AI transformation for ServiceNow customers.More NIM Agent Blueprints are in development for creating generative AI applications for customer service, content generation, software engineering, retail shopping advisors and R&D. NVIDIA plans to introduce new NIM Agent Blueprints monthly.NVIDIA Ecosystem Supercharges Enterprise AdoptionNVIDIAs partner ecosystem including global systems integrators and service delivery partners Accenture, Deloitte, SoftServe, Quantiphi and World Wide Technology are bringing NIM Agent Blueprints to the worlds enterprises.NIM Agent Blueprints can be optimized using customer interaction data with tools from NVIDIAs ecosystem of partners, such as Dataiku and DataRobot for model fine-tuning, governance, and monitoring, deepset, LlamaIndex and Langchain for building workflows, Weights and Biases for generative AI application evaluations, and CrowdStrike, Datadog, Fiddler AI, New Relic and Trend Micro for additional safeguarding. Infrastructure platform providers, including Nutanix, Red Hat and Broadcom, will support NIM Agent Blueprints on their enterprise solutions.Customers can build and deploy NIM Agent Blueprints on NVIDIA-Certified Systems from manufacturers such as Cisco, Dell Technologies, Hewlett Packard Enterprise and Lenovo, as well as on NVIDIA-accelerated cloud instances from Amazon Web Services, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure.To help enterprises put their data to work in generative AI applications, NIM Agent Blueprints can integrate with data and storage platforms from NVIDIA partners, such as Cohesity, Datastax, Dropbox, NetApp and VAST Data.A Collaborative Future for Developers and Data ScientistsGenerative AI is now fostering collaboration between developers and data scientists. Developers use NIM Agent Blueprints as a foundation to build their applications, while data scientists implement the data flywheel to continually improve their custom NIM microservices. As a NIM improves, so do the related applications, creating a cycle of continuous enhancement and data generation.With NIM Agent Blueprints and support from NVIDIAs partners virtually every enterprise can seamlessly integrate generative AI into their applications to drive efficiency and innovation across industries.Enterprises can experience NVIDIA NIM Agent Blueprints today.See notice regarding software product information.

Aiming at making the process faster and smarter, NVIDIA on Wednesday released the NIM Agent Blueprint for generative AI-based virtual screening.This innovative approach will reduce the time and cost of developing life-saving drugs, enabling quicker access to critical treatments for patients.This NIM Agent Blueprint introduces a paradigm shift in the drug discovery process, particularly in the crucial hit-to-lead transition, by moving from traditional fixed database screening to generative AI-driven molecule design and pre-optimization, enabling researchers to design better molecules faster.Whats a NIM? Whats a NIM Agent Blueprint?NVIDIA NIM microservices are modular, cloud-native components that accelerate AI model deployment and execution. These microservices allow researchers to integrate and scale advanced AI models within their workflows, enabling faster and more efficient processing of complex data.The NIM Agent Blueprint, a comprehensive guide, shows how these microservices can optimize key stages of drug discovery, such as hit identification and lead optimization.How Are They Used?Drug discovery is a complex process with three critical stages: target identification, hit identification and lead optimization. Target identification involves choosing the right biology to modify to treat the disease; hit identification is identifying potential molecules that will bind to that target; and lead optimization is improving the design of those molecules to be safer and more effective.This NVIDIA NIM Agent Blueprint, called generative virtual screening for accelerated drug discovery, identifies and improves virtual hits in a smarter and more efficient way.At its core are three essential AI models, now including the recently integrated AlphaFold2 as part of NVIDIAs NIM microservices.AlphaFold2, renowned for its groundbreaking impact on protein structure prediction, is now available as an NVIDIA NIM.MolMIM is a novel model developed by NVIDIA that generates molecules while simultaneously optimizing for multiple properties, such as high solubility and low toxicity.DiffDock is an advanced tool for quickly modeling the binding of small molecules to their protein targets.These models work in concert to improve the hit-to-lead process, making it more efficient and faster.Each of these AI models is packaged within NVIDIA NIM microservices portable containers designed to accelerate the performance, shorten time-to-market and simplify the deployment of generative AI models anywhere.The NIM Agent Blueprint integrates these microservices into a flexible, scalable, generative AI workflow that can help transform drug discovery.Leading computational drug discovery and biotechnology software providers that are using NIM microservices now, such as Benchling, Dotmatics, Terray, TetraScience and Cadence Molecular Sciences (OpenEye), are using NIM Agent Blueprints in their computer-aided drug discovery platforms.These integrations aim to make the hit-to-lead process faster and more intelligent, leading to the identification of more viable drug candidates in less time and at lower cost.Leading computational drug discovery and biotechnology software providers that are using NIM microservices now, such as Schrdinger, Benchling, Dotmatics, Terray, TetraScience and Cadence Molecular Sciences (OpenEye), are using NIM Agent Blueprints in their computer-aided drug discovery platforms.These integrations aim to make the hit-to-lead process faster and more intelligent, leading to the identification of more viable drug candidates in less time and at lower cost.Global professional services company Accenture is poised to tailor the NIM Agent Blueprint to the specific needs of drug development programs by optimizing the molecule generation step with input from pharmaceutical partners to inform the MolMIM NIM.In addition, the NIM microservices that comprise the NIM Agent Blueprint will soon be available on AWS HealthOmics, a purpose-built service that helps customers orchestrate biological analyses. This includes streamlining the integration of AI into existing drug discovery workflows.Revolutionizing Drug Development With AIThe stakes in drug discovery are high.Developing a new drug typically costs around $2.6 billion and can take 10-15 years, with a success rate of less than 10%.By making molecular design smarter with NVIDIAs AI-powered NIM Agent Blueprint, pharmaceutical companies can reduce these costs and shorten development timelines in the $1.5 trillion global pharmaceutical market.This NIM Agent Blueprint represents a significant shift from traditional drug discovery methods, offering a generative AI approach that pre-optimizes molecules for desired therapeutic properties.For example, MolMIM, the generative model for molecules within this NIM Agent Blueprint, uses advanced functions to steer the generation of molecules with optimized pharmacokinetic properties such as absorption rate, protein binding, half-life and other properties a marked advancement over previous methods.This smarter approach to small molecule design enhances the potential for successful lead optimization, accelerating the overall drug discovery process.This leap in technology could lead to faster, more targeted treatments, addressing growing challenges in healthcare, from rising costs to an aging population.NVIDIAs commitment to supporting researchers with the latest advancements in accelerated computing underscores its role in solving the most complex problems in drug discovery.Visit build.nvidia.com to download the NIM Agent Blueprint for generative AI-based virtual screening and take the first step toward faster, more efficient drug development.See notice regarding software product information.

Nations around the world are pursuing sovereign AI to produce artificial intelligence using their own computing infrastructure, data, workforce and business networks to ensure AI systems align with local values, laws and interests.In support of these efforts, NVIDIA today announced the availability of four new NVIDIA NIM microservices that enable developers to more easily build and deploy high-performing generative AI applications.The microservices support popular community models tailored to meet regional needs. They enhance user interactions through accurate understanding and improved responses based on local languages and cultural heritage.In the Asia-Pacific region alone, generative AI software revenue is expected to reach $48 billion by 2030 up from $5 billion this year, according to ABI Research.Llama-3-Swallow-70B, trained on Japanese data, and Llama-3-Taiwan-70B, trained on Mandarin data, are regional language models that provide a deeper understanding of local laws, regulations and other customs.The RakutenAI 7B family of models, built on Mistral-7B, were trained on English and Japanese datasets, and are available as two different NIM microservices for Chat and Instruct. Rakutens foundation and instruct models have achieved leading scores among open Japanese large language models, landing the top average score in the LM Evaluation Harness benchmark carried out from January to March 2024.Training a large language model (LLM) on regional languages enhances the effectiveness of its outputs by ensuring more accurate and nuanced communication, as it better understands and reflects cultural and linguistic subtleties.The models offer leading performance for Japanese and Mandarin language understanding, regional legal tasks, question-answering, and language translation and summarization compared with base LLMs like Llama 3.Nations worldwide from Singapore, the United Arab Emirates, South Korea and Sweden to France, Italy and India are investing in sovereign AI infrastructure.The new NIM microservices allow businesses, government agencies and universities to host native LLMs in their own environments, enabling developers to build advanced copilots, chatbots and AI assistants.Developing Applications With Sovereign AI NIM MicroservicesDevelopers can easily deploy the sovereign AI models, packaged as NIM microservices, into production while achieving improved performance.The microservices, available with NVIDIA AI Enterprise, are optimized for inference with the NVIDIA TensorRT-LLM open-source library.NIM microservices for Llama 3 70B which was used as the base model for the new Llama3-Swallow-70B and Llama-3-Taiwan-70B NIM microservices can provide up to 5x higher throughput. This lowers the total cost of running the models in production and provides better user experiences by decreasing latency.The new NIM microservices are available today as hosted application programming interfaces (APIs).Tapping NVIDIA NIM for Faster, More Accurate Generative AI OutcomesThe NIM microservices accelerate deployments, enhance overall performance and provide the necessary security for organizations across global industries, including healthcare, finance, manufacturing, education and legal.The Tokyo Institute of Technology fine-tuned Llama-3-Swallow 70B using Japanese-language data.LLMs are not mechanical tools that provide the same benefit for everyone. They are rather intellectual tools that interact with human culture and creativity. The influence is mutual where not only are the models affected by the data we train on, but also our culture and the data we generate will be influenced by LLMs, said Rio Yokota, professor at the Global Scientific Information and Computing Center at the Tokyo Institute of Technology. Therefore, it is of paramount importance to develop sovereign AI models that adhere to our cultural norms. The availability of Llama-3-Swallow as an NVIDIA NIM microservice will allow developers to easily access and deploy the model for Japanese applications across various industries.For instance, a Japanese AI company, Preferred Networks, uses the model to develop a healthcare specific model trained on a unique corpus of Japanese medical data, called Llama3-Preferred-MedSwallow-70B, that tops scores on the Japan National Examination for Physicians.Chang Gung Memorial Hospital (CGMH), one of the leading hospitals in Taiwan, is building a custom-made AI Inference Service (AIIS) to centralize all LLM applications within the hospital system. Using Llama 3-Taiwan 70B, it is improving the efficiency of frontline medical staff with more nuanced medical language that patients can understand.By providing instant, context-appropriate guidance, AI applications built with local-language LLMs streamline workflows and serve as a continuous learning tool to support staff development and improve the quality of patient care, said Dr. Changfu Kuo, director of the Center for Artificial Intelligence in Medicine at CGMH, Linko Branch. NVIDIA NIM is simplifying the development of these applications, allowing for easy access and deployment of models trained on regional languages with minimal engineering expertise.Taiwan-based Pegatron, a maker of electronic devices, will adopt the Llama 3-Taiwan 70B NIM microservice for internal- and external-facing applications. It has integrated it with its PEGAAi Agentic AI System to automate processes, boosting efficiency in manufacturing and operations.Llama-3-Taiwan 70B NIM is also being used by global petrochemical manufacturer Chang Chun Group, world-leading printed circuit board company Unimicron, technology-focused media company TechOrange, online contract service company LegalSign.ai and generative AI startup APMIC. These companies are also collaborating on the open model.Creating Custom Enterprise Models With NVIDIA AI FoundryWhile regional AI models can provide culturally nuanced and localized responses, enterprises still need to fine-tune them for their business processes and domain expertise.NVIDIA AI Foundry is a platform and service that includes popular foundation models, NVIDIA NeMo for fine-tuning, and dedicated capacity on NVIDIA DGX Cloud to provide developers a full-stack solution for creating a customized foundation model packaged as a NIM microservice.Additionally, developers using NVIDIA AI Foundry have access to the NVIDIA AI Enterprise software platform, which provides security, stability and support for production deployments.NVIDIA AI Foundry gives developers the necessary tools to more quickly and easily build and deploy their own custom, regional language NIM microservices to power AI applications, ensuring culturally and linguistically appropriate results for their users.

News summary: New libraries in accelerated computing deliver order-of-magnitude speedups and reduce energy consumption and costs in data processing, generative AI, recommender systems, AI data curation, data processing, 6G research, AI-physics and more. They include:LLM applications: NeMo Curator, to create custom datasets, adds image curation and Nemotron-4 340B for high-quality synthetic data generationData processing: cuVS for vector search to build indexes in minutes instead of days and a new Polars GPU Engine in open betaPhysical AI: For physics simulation, Warp accelerates computations with a new TIle API. For wireless network simulation, Aerial adds more map formats for ray tracing and simulation. And for link-level wireless simulation, Sionna adds a new toolchain for real-time inferenceCompanies around the world are increasingly turning to NVIDIA accelerated computing to speed up applications they first ran on CPUs only. This has enabled them to achieve extreme speedups and benefit from incredible energy savings.In Houston, CPFD makes computational fluid dynamics simulation software for industrial applications, like its Barracuda Virtual Reactor software that helps design next-generation recycling facilities. Plastic recycling facilities run CPFD software in cloud instances powered by NVIDIA accelerated computing. With a CUDA GPU-accelerated virtual machine, they can efficiently scale and run simulations 400x faster and 140x more energy efficiently than using a CPU-based workstation.Bottles being loaded into a plastics recycling facility. AI-generated image.A popular video conferencing application captions several hundred thousand virtual meetings an hour. When using CPUs to create live captions, the app could query a transformer-powered speech recognition AI model three times a second. After migrating to GPUs in the cloud, the applications throughput increased to 200 queries per second a 66x speedup and 25x energy-efficiency improvement.In homes across the globe, an e-commerce website connects hundreds of millions of shoppers a day to the products they need using an advanced recommendation system powered by a deep learning model, running on its NVIDIA accelerated cloud computing system. After switching from CPUs to GPUs in the cloud, it achieved significantly lower latency with a 33x speedup and nearly 12x energy-efficiency improvement.With the exponential growth of data, accelerated computing in the cloud is set to enable even more innovative use cases.NVIDIA Accelerated Computing on CUDA GPUs Is Sustainable ComputingNVIDIA estimates that if all AI, HPC and data analytics workloads that are still running on CPU servers were CUDA GPU-accelerated, data centers would save 40 terawatt-hours of energy annually. Thats the equivalent energy consumption of 5 million U.S. homes per year.Accelerated computing uses the parallel processing capabilities of CUDA GPUs to complete jobs orders of magnitude faster than CPUs, improving productivity while dramatically reducing cost and energy consumption.Although adding GPUs to a CPU-only server increases peak power, GPU acceleration finishes tasks quickly and then enters a low-power state. The total energy consumed with GPU-accelerated computing is significantly lower than with general-purpose CPUs, while yielding superior performance.GPUs achieve 20x greater energy efficiency compared to traditional computing on CPU-only servers because they deliver greater performance per watt, completing more tasks in less time.In the past decade, NVIDIA AI computing has achieved approximately 100,000x more energy efficiency when processing large language models. To put that into perspective, if the efficiency of cars improved as much as NVIDIA has advanced the efficiency of AI on its accelerated computing platform, theyd get 500,000 miles per gallon. Thats enough to drive to the moon, and back, on less than a gallon of gasoline.In addition to these dramatic boosts in efficiency on AI workloads, GPU computing can achieve incredible speedups over CPUs. Customers of the NVIDIA accelerated computing platform running workloads on cloud service providers saw speedups of 10-180x across a gamut of real-world tasks, from data processing to computer vision, as the chart below shows.Speedups of 10-180x achieved in real-world implementations by cloud customers across workloads with the NVIDIA accelerated computing platform.As workloads continue to demand exponentially more computing power, CPUs have struggled to provide the necessary performance, creating a growing performance gap and driving compute inflation. The chart below illustrates a multiyear trend of how data growth has far outpaced the growth in compute performance per watt of CPUs.The widening gap between data growth and the lagging compute performance per watt of CPUs.The energy savings of GPU acceleration frees up what would otherwise have been wasted cost and energy.With its massive energy-efficiency savings, accelerated computing is sustainable computing.The Right Tools for Every JobGPUs cannot accelerate software written for general-purpose CPUs. Specialized algorithm software libraries are needed to accelerate specific workloads. Just like a mechanic would have an entire toolbox from a screwdriver to a wrench for different tasks, NVIDIA provides a diverse set of libraries to perform low-level functions like parsing and executing calculations on data.Each NVIDIA CUDA library is optimized to harness hardware features specific to NVIDIA GPUs. Combined, they encompass the power of the NVIDIA platform.New updates continue to be added on the CUDA platform roadmap, expanding across diverse use cases:LLM ApplicationsNeMo Curator gives developers the flexibility to quickly create custom datasets in large language model (LLM) use cases. Recently, we announced capabilities beyond text to expand to multimodal support, including image curation.SDG (synthetic data generation) augments existing datasets with high-quality, synthetically generated data to customize and fine-tune models and LLM applications. We announced Nemotron-4 340B, a new suite of models specifically built for SDG that enables businesses and developers to use model outputs and build custom models.Data Processing ApplicationscuVS is an open-source library for GPU-accelerated vector search and clustering that delivers incredible speed and efficiency across LLMs and semantic search. The latest cuVS allows large indexes to be built in minutes instead of hours or even days, and searches them at scale.Polars is an open-source library that makes use of query optimizations and other techniques to process hundreds of millions of rows of data efficiently on a single machine. A new Polars GPU engine powered by NVIDIAs cuDF library will be available in open beta. It delivers up to a 10x performance boost compared to CPU, bringing the energy savings of accelerated computing to data practitioners and their applications.Physical AIWarp, for high-performance GPU simulation and graphics, helps accelerate spatial computing by making it easier to write differentiable programs for physics simulation, perception, robotics and geometry processing. The next release will have support for a new Tile API that allows developers to use Tensor Cores inside GPUs for matrix and Fourier computations.Aerial is a suite of accelerated computing platforms that includes Aerial CUDA-Accelerated RAN and Aerial Omniverse Digital Twin for designing, simulating and operating wireless networks for commercial applications and industry research. The next release will include a new expansion of Aerial with more map formats for ray tracing and simulations with higher accuracy.Sionna is a GPU-accelerated open-source library for link-level simulations of wireless and optical communication systems. With GPUs, Sionna achieves orders-of-magnitude faster simulation, enabling interactive exploration of these systems and paving the way for next-generation physical layer research. The next release will include the entire toolchain required to design, train and evaluate neural network-based receivers, including support for real-time inference of such neural receivers using NVIDIA TensorRT.NVIDIA provides over 400 libraries. Some, like CV-CUDA, excel at pre- and post-processing of computer vision tasks common in user-generated video, recommender systems, mapping and video conferencing. Others, like cuDF, accelerate data frames and tables central to SQL databases and pandas in data science.CAD Computer-Aided Design, CAE Computer-Aided Engineering, EDA Electronic Design AutomationMany of these libraries are versatile for example, cuBLAS for linear algebra acceleration and can be used across multiple workloads, while others are highly specialized to focus on a specific use case, like cuLitho for silicon computational lithography.For researchers who dont want to build their own pipelines with NVIDIA CUDA-X libraries, NVIDIA NIM provides a streamlined path to production deployment by packaging multiple libraries and AI models into optimized containers. The containerized microservices deliver improved throughput out of the box.Augmenting these libraries performance are an expanding number of hardware-based acceleration features that deliver speedups with the highest energy efficiencies. The NVIDIA Blackwell platform, for example, includes a decompression engine that unpacks compressed data files inline up to 18x faster than CPUs. This dramatically accelerates data processing applications that need to frequently access compressed files in storage like SQL, Apache Spark and pandas, and decompress them for runtime computation.The integration of NVIDIAs specialized CUDA GPU-accelerated libraries into cloud computing platforms delivers remarkable speed and energy efficiency across a wide range of workloads. This combination drives significant cost savings for businesses and plays a crucial role in advancing sustainable computing, helping billions of users relying on cloud-based workloads to benefit from a more sustainable and cost-effective digital ecosystem.Learn more about NVIDIAs sustainable computing efforts and check out the Energy Efficiency Calculator to discover potential energy and emissions savings.See notice regarding software product information.

A deep technology conference for processor and system architects from industry and academia has become a key forum for the trillion-dollar data center computing market.At Hot Chips 2024 next week, senior NVIDIA engineers will present the latest advancements powering the NVIDIA Blackwell platform, plus research on liquid cooling for data centers and AI agents for chip design.Theyll share how:NVIDIA Blackwell brings together multiple chips, systems and NVIDIA CUDA software to power the next generation of AI across use cases, industries and countries.NVIDIA GB200 NVL72 a multi-node, liquid-cooled, rack-scale solution that connects 72 Blackwell GPUs and 36 Grace CPUs raises the bar for AI system design.NVLink interconnect technology provides all-to-all GPU communication, enabling record high throughput and low-latency inference for generative AI.The NVIDIA Quasar Quantization System pushes the limits of physics to accelerate AI computing.NVIDIA researchers are building AI models that help build processors for AI.An NVIDIA Blackwell talk, taking place Monday, Aug. 26, will also spotlight new architectural details and examples of generative AI models running on Blackwell silicon.Its preceded by three tutorials on Sunday, Aug. 25, that will cover how hybrid liquid-cooling solutions can help data centers transition to more energy-efficient infrastructure and how AI models, including large language model (LLM)-powered agents, can help engineers design the next generation of processors.Together, these presentations showcase the ways NVIDIA engineers are innovating across every area of data center computing and design to deliver unprecedented performance, efficiency and optimization.Be Ready for BlackwellNVIDIA Blackwell is the ultimate full-stack computing challenge. It comprises multiple NVIDIA chips, including the Blackwell GPU, Grace CPU, BlueField data processing unit, ConnectX network interface card, NVLink Switch, Spectrum Ethernet switch and Quantum InfiniBand switch.Ajay Tirumala and Raymond Wong, directors of architecture at NVIDIA, will provide a first look at the platform and explain how these technologies work together to deliver a new standard for AI and accelerated computing performance while advancing energy efficiency.The multi-node NVIDIA GB200 NVL72 solution is a perfect example. LLM inference requires low-latency, high-throughput token generation. GB200 NVL72 acts as a unified system to deliver up to 30x faster inference for LLM workloads, unlocking the ability to run trillion-parameter models in real time.Tirumala and Wong will also discuss how the NVIDIA Quasar Quantization System which brings together algorithmic innovations, NVIDIA software libraries and tools, and Blackwells second-generation Transformer Engine supports high accuracy on low-precision models, highlighting examples using LLMs and visual generative AI.Keeping Data Centers CoolThe traditional hum of air-cooled data centers may become a relic of the past as researchers develop more efficient and sustainable solutions that use hybrid cooling, a combination of air and liquid cooling.Liquid-cooling techniques move heat away from systems more efficiently than air, making it easier for computing systems to stay cool even while processing large workloads. The equipment for liquid cooling also takes up less space and consumes less power than air-cooling systems, allowing data centers to add more server racks and therefore more compute power in their facilities.Ali Heydari, director of data center cooling and infrastructure at NVIDIA, will present several designs for hybrid-cooled data centers.Some designs retrofit existing air-cooled data centers with liquid-cooling units, offering a quick and easy solution to add liquid-cooling capabilities to existing racks. Other designs require the installation of piping for direct-to-chip liquid cooling using cooling distribution units or by entirely submerging servers in immersion cooling tanks. Although these options demand a larger upfront investment, they lead to substantial savings in both energy consumption and operational costs.Heydari will also share his teams work as part of COOLERCHIPS, a U.S. Department of Energy program to develop advanced data center cooling technologies. As part of the project, the team is using the NVIDIA Omniverse platform to create physics-informed digital twins that will help them model energy consumption and cooling efficiency to optimize their data center designs.AI Agents Chip In for Processor DesignSemiconductor design is a mammoth challenge at microscopic scale. Engineers developing cutting-edge processors work to fit as much computing power as they can onto a piece of silicon a few inches across, testing the limits of whats physically possible.AI models are supporting their work by improving design quality and productivity, boosting the efficiency of manual processes and automating some time-consuming tasks. The models include prediction and optimization tools to help engineers rapidly analyze and improve designs, as well as LLMs that can assist engineers with answering questions, generating code, debugging design problems and more.Mark Ren, director of design automation research at NVIDIA, will provide an overview of these models and their uses in a tutorial. In a second session, hell focus on agent-based AI systems for chip design.AI agents powered by LLMs can be directed to complete tasks autonomously, unlocking broad applications across industries. In microprocessor design, NVIDIA researchers are developing agent-based systems that can reason and take action using customized circuit design tools, interact with experienced designers, and learn from a database of human and agent experiences.NVIDIA experts arent just building this technology theyre using it. Ren will share examples of how engineers can use AI agents for timing report analysis, cell cluster optimization processes and code generation. The cell cluster optimization work recently won best paper at the first IEEE International Workshop on LLM-Aided Design.Register for Hot Chips, taking place Aug. 25-27, at Stanford University and online.

Snowflake is using AI to help enterprises transform data into insights and applications. In this episode of NVIDIAs AI Podcast, host Noah Kravitz and Baris Gultekin, head of AI at Snowflake, discuss how the companys AI Data Cloud platform enables customers to access and manage data at scale. By separating the storage of data from compute, Snowflake has allowed organizations across the world to connect via cloud technology and work on a unified platform eliminating data silos and streamlining collaborative workflows.The AI Podcast Snowflakes Baris Gultekin on Unlocking the Value of Data With Large Language Models Ep. 231Time Stamps1:45: What does Snowflake do?3:18: Snowflakes AI and data strategies building a platform with natural language analysis5:30: How to efficiently access large language models with Snowflake Cortex11:49: Snowflakes open-source LLM: Arctic16:18: Gultekins journey in AI and data science23:05: The AI industry in three to five years real-world applications of Snowflake technology27:54: Gutlekins advice for professionals interested in AIYou Might Also Like:How Roblox Uses Generative AI to Enhance User Experiences Ep. 227Roblox is a colorful online platform reimagining the way people come together. Anupam Singh, vice president of AI and growth engineering at Roblox, discusses how the company uses generative AI to enhance virtual experiences and bolster inclusivity and user safety.NVIDIAs Jim Fan Delves Into Large Language Models and Their Industry Impact Ep. 204Most know Minecraft as the popular blocky sandbox game, but for Jim Fan, senior AI scientist at NVIDIA, Minecraft was the perfect place to test the decision-making agency of AI models. Fan discusses how he used large language models to research open-ended AI agents and create Voyager, an AI bot built with Chat GPT-4 that can autonomously play Minecraft.Media.Monks Lewis Smithingham on Enhancing Media and Marketing With AI Ep. 222Media.Monks platform Wormhole streamlines marketing and content creation workflows with AI-powered insights. Lewis Smithingham, senior vice president of innovation and special operations at Media.Monks, addresses AIs potential in the entertainment and advertisement industries.NVIDIAs Annamali Chockalingam on the Rise of LLMs Ep. 206LLMs are in the spotlight, capable of tasks like generation, summarization, translation, instruction and chatting. Annamalai Chockalingam, senior product manager of developer marketing at NVIDIA, discusses how a combination of these modalities and actions can build applications to solve any problem.Subscribe to the AI PodcastGet the AI Podcast through iTunes, Google Play, Amazon Music, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn.Make the AI Podcast better: Have a few minutes to spare? Fill out this listener survey.