Physics, Simulation, and the Dirty Secret of AI Training: A Conversation with Dov (Dubi) Katz

The CTO of Formic Robotics has spent much of the past year focused on the gap between simulators and the real world. We asked him why that gap matters so much—and why he is still doing the technical training himself.

There is a tension at the heart of modern robotics AI, and most people outside the field do not see it. Almost all the training that produces capable robotic policies occurs in simulation. Testing whether those policies are actually any good has to happen in the real world. The gap between those two is currently one of the hardest problems in applied AI.

Dov (Dubi) Katz, the CTO of Formic Robotics, has spent much of his time on that gap. He recently completed NVIDIA training focused on physics simulation, which is not a typical use of a CTO's time, and which signals something about where he thinks the work actually is. We sat down with him to dig into it.

Let's start with the obvious question. Why did simulation become the default for training robotic AI in the first place?

Because training a robot in the real world is slow and expensive. Every trial requires a physical system that can be damaged, a safety envelope that has to be maintained, and a human team to run the experiment. The data you collect from an afternoon of real-world trials is often less than what a single GPU can produce in a minute of simulation. For AI systems that need millions of trajectories to converge, real-world training is essentially a non-starter.

So the field moved to simulation. In the simulator, a robot can fall over ten thousand times in an hour, and none of it costs anything. Environments can be procedurally generated. Sensor noise can be parameterized. Lighting, friction, and physics constants can all be varied across runs to expose the learning algorithm to a broader distribution than any real facility could provide.

And it has worked, to a point.

It has worked impressively. Many of the capabilities that now look routine in robotics—legged locomotion over uneven terrain, dexterous manipulation, visual serving on moving targets—emerged from simulation-heavy training pipelines. The paradigm is genuinely powerful. The problem is that it only works up to a certain point.

That point being the "sim-to-real gap" people talk about. What is that, exactly?

Sim-to-real transfer is the technical term for what happens when a policy trained in simulation is deployed on actual hardware in the real world. Sometimes the transfer is clean, and the robot performs roughly the way it did in the simulator. More often, something has shifted, and the performance degrades in ways that are hard to diagnose.

The sources of that degradation are numerous. The physics of the simulator is always an approximation. Contact dynamics, friction, and compliance are notoriously hard to get right, and small errors compound when you are simulating thousands of interactions in sequence. The sensor model in the simulator rarely captures the full spectrum of real sensors—how lighting affects a camera at the edges of its dynamic range, how a depth sensor behaves near reflective surfaces, all of that.

Is this a problem that goes away with better tooling, or is it more fundamental than that?

I have come around to the view that the gap is not a nuisance to be engineered away. It is a structural feature of training systems in virtual environments and then deploying them in physical ones. The right question is not how to eliminate the gap, because that is not possible. The right question is how to train in a way that produces policies robust to the gap when it inevitably appears.

What approaches are being tried, and how do you think about them?

They tend to be complementary rather than competing. Domain randomization varies the simulator parameters across training runs, exposing the policy to a wide enough range of conditions that it learns to be robust rather than specialized. Better physics engines aim to narrow the gap itself by being more accurate. System identification aims to estimate the real-world parameters of a specific robot and its environment, then tune the simulator to match. Learning-based approaches to simulation itself, including neural surrogates for physics, try to close the gap from a different direction entirely.

The interesting thing about the current moment is that several of these are becoming usable simultaneously. NVIDIA's physics simulation tooling, which is what I have been training on, represents a bet on high-fidelity GPU-accelerated physics that can run fast enough to support the large-scale training regimes modern AI requires. The bet is that if the simulator is accurate enough and fast enough, much of the transfer problem becomes easier, and the remaining gap can be addressed with domain randomization and system identification.

Whether that bet fully pays off is still an open question. But the fact that a simulator can now run physics at a fidelity that was impossible a few years ago is a genuine change in the kinds of robotic policies that are now trainable.

Let me ask the more personal question. You are a CTO. Doing NVIDIA training in physics simulation is not what most CTOs are doing with their time. Why are you still in there?

It would be entirely defensible to delegate the details to my team and focus on the roadmap and hiring. I do not because the problems we are working on are at the frontier, and leading from a distance does not work in a field that is moving this fast. Staying close to the technical reality—the actual capabilities of the simulators, the actual behavior of the physics, the actual failure modes of the policies—is how I maintain the kind of judgment that lets me make good decisions about where to invest.

There is also something more philosophical in it, honestly. The sim-to-real gap is, at heart, a humility problem. The simulator will always be wrong in ways you did not anticipate. The real world is always going to surprise you. Building systems that handle that gracefully requires engineers and leaders who have internalized the lesson viscerally, not just intellectually. I am not sure you can get there if you stop doing the work.