NVIDIA has launched an extensive collection of open source agent tools and technologies designed to automate the development of physical AI systems used in robots, autonomous vehicles, factory inspection lines, and hospital automation.
This release deploys NVIDIA’s existing hardware and simulation platform stack, including Omniverse, Isaac, Cosmos, Metropolis, Alpamayo, and Jetson, to be directly invoked by AI coding agents.
The most practical effect is that workflows that previously required significant manual engineering effort can now be orchestrated and executed autonomously by AI agents.
“AI agents are revolutionizing software development, and this transformation is now coming to physical AI and expanding to systems that are transforming transportation, manufacturing, healthcare and robotics,” he said. Jensen Huang, founder and CEO of NVIDIA.
“Enabling agents to directly use NVIDIA libraries, models, and frameworks will accelerate physical AI development, allowing developers to build the robots, autonomous vehicles, and industrial systems of the future at incredible speeds.”
From code generation to physical orchestration
Until recently, AI agents have primarily worked in the software domain, including writing code, summarizing documents, and answering queries.
NVIDIA’s announcement marks the industry’s push toward agents that can manage much more complex, multi-step technological processes in the physical world.
The new technology, an optimized and repeatable package of instructions, tells agents which tools to call, what outputs to produce, and how to validate the results throughout the entire physical AI development pipeline.
This includes generating synthetic training data, running simulations, fine-tuning models, automating labeling, and managing deployment to edge hardware.
For developers building robots or autonomous systems, this means significantly narrowing the gap between a working prototype and a production-ready, continuously improving system.
Instead of manually configuring each step of the pipeline, teams can instruct agents to handle orchestration and engineers can focus on higher-level design and validation decisions.
Omniverse at the heart of the Workspace AI story
Omniverse, NVIDIA’s platform for building and simulating industrial digital twins, is at the core of several key use cases announced with the toolkit.
Industrial software companies including Cadence, Dassault Systèmes, Siemens, and Synopsys are using Omniverse libraries and agent technology for engineering data inspection, simulation, and interactive digital twins.
PTC and others are using it in conjunction with OpenUSD-based workflows to transform CAD data into simulation-ready environments.
This means that digital twins, long discussed as a future-state concept in enterprise technology circles, are becoming active, agent-centric workplaces in their own right.
Physical spaces such as semiconductor factories, hospital wards, and manufacturing floors are modeled, simulated, and optimized through AI before any actual changes are made. For example, SK Hynix is using Omniverse to build a semiconductor fab digital twin as part of its Autonomous Fab 2030 roadmap.
This is immersive spatial computing technology applied to the most demanding operating environments in the industry, not consumer entertainment.
Real results are already being seen.
Nvidia has reported a series of performance numbers from early adopters that give us an idea of how the tooling is used in practice.
Electronics manufacturer Pegatron reported a 67% reduction in model training and deployment times by using NVIDIA’s defect image generation technology to generate synthetic training data for visual inspection systems, according to the company.
Delta Electronics used the same technology to improve defect detection rates by 17% on a metal busbar soldering line.
Foxconn also reportedly achieved a 3% improvement in first-pass yield from its manufacturing line. Inventec says it has reduced the defect data collection effort required for laptop chassis manufacturing by 30 percent.
In the autonomous vehicle space, Li Auto, Afari, and DeepRoute.ai are using NVIDIA Omniverse NuRec models to generate more than 300,000 renderings and simulations per day and accelerate training and evaluation of AV systems.
What this means for IT and technology leaders
If you’re an IT and technology leader in an enterprise environment, it’s worth taking a closer look at NVIDIA’s announcement.
The change here is from AI as a productivity layer on top of existing workflows to AI as an active orchestrator of technology infrastructure.
When agents can autonomously manage simulation pipelines, fine-tune models, and deploy to edge hardware, the governance, security, and integration issues that IT teams are already navigating using software-side AI tools become even more complex.
NVIDIA included security and governance tools in the release. NemoClaw blueprints and OpenShell runtime provide policy-based security and privacy controls for local or cloud deployments. However, large-scale enterprise adoption will require IT teams to think carefully about how autonomous physical AI workflows fit into their existing operational frameworks.
The toolkit is now available through GitHub and Skill.sh with cloud integrations from Microsoft, CoreWeave, and Nebius. You can try out a pre-configured environment for synthetic data generation on NVIDIA Brev.