Teaching autonomous robots and vehicles how to interact with the physical world requires vast amounts of high-quality data. To give researchers and developers a head start, NVIDIA is releasing a massive, open-source dataset for building the next generation of physical AI.Announced at NVIDIA GTC, a global AI conference taking place this week in San Jose, California, this commercial-grade, pre-validated dataset can help researchers and developers kickstart physical AI projects that can be prohibitively difficult to start from scratch. Developers can either directly use the dataset for model pretraining, testing and validation or use it during post-training to fine-tune world foundation models, accelerating the path to deployment.The initial dataset is now available on Hugging Face, offering developers 15 terabytes of data representing more than 320,000 trajectories for robotics training, plus up to 1,000 Universal Scene Description (OpenUSD) assets, including a SimReady collection. Dedicated data to support end-to-end autonomous vehicle (AV) development which will include 20-second clips of diverse traffic scenarios spanning over 1,000 cities across the U.S. and two dozen European countries is coming soon.The NVIDIA Physical AI Dataset includes hundreds of SimReady assets for rich scenario building.This dataset will grow over time to become the worlds largest unified and open dataset for physical AI development. It could be applied to develop AI models to power robots that safely maneuver warehouse environments, humanoid robots that support surgeons during procedures and AVs that can navigate complex traffic scenarios like construction zones.The NVIDIA Physical AI Dataset is slated to contain a subset of the real-world and synthetic data NVIDIA uses to train, test and validate physical AI for the NVIDIA Cosmos world model development platform, the NVIDIA DRIVE AV software stack, the NVIDIA Isaac AI robot development platform and the NVIDIA Metropolis application framework for smart cities.Early adopters include the Berkeley DeepDrive Center at the University of California, Berkeley, the Carnegie Mellon Safe AI Lab and the Contextual Robotics Institute at University of California, San Diego.We can do a lot of things with this dataset, such as training predictive AI models that help autonomous vehicles better track the movements of vulnerable road users like pedestrians to improve safety, said Henrik Christensen, director of multiple robotics and autonomous vehicle labs at UCSD. A dataset that provides a diverse set of environments and longer clips than existing open-source resources will be tremendously helpful to advance robotics and AV research.Addressing the Need for Physical AI DataThe NVIDIA Physical AI Dataset can help developers scale AI performance during pretraining, where more data helps build a more robust model and during post-training, where an AI model is trained on additional data to improve its performance for a specific use case.Collecting, curating and annotating a dataset that covers diverse scenarios and accurately represents the physics and variation of the real world is time-consuming, presenting a bottleneck for most developers. For academic researchers and small enterprises, running a fleet of vehicles over months to gather data for autonomous vehicle AI is impractical and costly and, since much of the footage collected is uneventful, typically just 10% of data is used for training.But this scale of data collection is essential to building safe, accurate, commercial-grade models. NVIDIA Isaac GR00T robotics models take thousands of hours of video clips for post-training the GR00T N1 model, for example, was trained on an expansive humanoid dataset of real and synthetic data. The NVIDIA DRIVE AV end-to-end AI model for autonomous vehicles requires tens of thousands of hours of driving data to develop.https://blogs.nvidia.com/wp-content/uploads/2025/03/rgb_5sec-1.mp4This open dataset, comprising thousands of hours of multicamera video at unprecedented diversity, scale and geography will particularly benefit the field of safety research by enabling new work on identifying outliers and assessing model generalization performance. The effort contributes to NVIDIA Halos full-stack AV safety system.In addition to harnessing the NVIDIA Physical AI Dataset to help meet their data needs, developers can further boost AI development with tools like NVIDIA NeMo Curator, which process vast datasets efficiently for model training and customization. Using NeMo Curator, 20 million hours of video can be processed in just two weeks on NVIDIA Blackwell GPUs, compared with 3.4 years on unoptimized CPU pipelines.Robotics developers can also tap the new NVIDIA Isaac GR00T blueprint for synthetic manipulation motion generation, a reference workflow built on NVIDIA Omniverse and NVIDIA Cosmos that uses a small number of human demonstrations to create massive amounts of synthetic motion trajectories for robot manipulation.University Labs Set to Adopt Dataset for AI DevelopmentThe robotics labs at UCSD include teams focused on medical applications, humanoids and in-home assistive technology. Christensen anticipates that the Physical AI Datasets robotics data could help develop semantic AI models that understand the context of spaces like homes, hotel rooms and hospitals.One of our goals is to achieve a level of understanding where, if a robot was asked to put your groceries away, it would know exactly which items should go in the fridge and what goes in the pantry, he said.In the field of autonomous vehicles, Christensens lab could apply the dataset to train AI models to understand the intention of various road users and predict the best action to take. His research teams could also use the dataset to support the development of digital twins that simulate edge cases and challenging weather conditions. These simulations could be used to train and test autonomous driving models in situations that are rare in real-world environments.At Berkeley DeepDrive, a leading research center on AI for autonomous systems, the dataset could support the development of policy models and world foundation models for autonomous vehicles.Data diversity is incredibly important to train foundation models, said Wei Zhan, codirector of Berkeley DeepDrive. This dataset could support state-of-the-art research for public and private sector teams developing AI models for autonomous vehicles and robotics.Researchers at Carnegie Mellon Universitys Safe AI Lab plan to use the dataset to advance their work evaluating and certifying the safety of self-driving cars. The team plans to test how a physical AI foundation model trained on this dataset performs in a simulation environment with rare conditions and compare its performance to an AV model trained on existing datasets.This dataset covers different types of roads and geographies, different infrastructure, different weather environments, said Ding Zhao, associate professor at CMU and head of the Safe AI Lab. Its diversity could be quite valuable in helping us train a model with causal reasoning capabilities in the physical world that understands edge cases and long-tail problems.Access the NVIDIA Physical AI dataset on Hugging Face. Build foundational knowledge with courses such as the Learn OpenUSD learning path and Robotics Fundamentals learning path. And to learn more about the latest advancements in physical AI, watch the GTC keynote by NVIDIA founder and CEO Jensen Huang.See notice regarding software product information.