We are living through an unprecedented moment in robotics. Frontier capabilities are advancing almost daily, fueled by foundation models, more powerful compute, richer sensors, better simulators, and an explosion of new robot platforms. But turning any of them into a system that reliably performs a real task still requires an enormous amount of manual engineering: Teams have to integrate hardware, calibrate sensors, build simulation environments, collect and clean data, adapt perception and control loops, design evaluations, diagnose failures on hardware, and repeat the entire loop until the system works. The capabilities are increasingly abundant. What remains scarce is the know-how to reliably deploy them. Building repeatable, autonomous deployment workflows requires bringing these capabilities into a common environment and distilling the engineering knowledge that today lives inside specialized robotics teams.
A parallel can be drawn to modern software engineering where the strength of combining powerful models with both the right harnesses and verifiable feedback has induced a paradigm shift. Coding harnesses give agents access to repositories, tools, execution feedback, and the ability to iteratively modify their work. A crucial component of these harnesses is the availability of scalable, verifiable feedback: tests, compilers, and benchmarks make it possible to objectively evaluate outputs and turn those evaluations into learning signals that improve the models themselves. This feedback loop has helped make software engineering one of the clearest success stories for modern AI.
Robotics needs its own harnesses, and over the last few months, we have built them into GRID. They give agents the tools and feedback to understand a robot’s capabilities, create the experience it needs to learn, develop skills, and make those skills work reliably in the real world. Together, they enable a closed engineering loop. When a task demands something GRID cannot yet do, the agent builds that capability and adds it back to the platform.
We call this Auto-Engineering.
Two years ago, we set out to capture the full breadth of robotics engineering in a single, programmable system. Our concentrated efforts culminated in a monorepo that integrates across 50+ OEM robots, hundreds of models and modern workflows across perception, control, planning, reinforcement and imitation learning, distributed inference, simulation, and deployment, covering virtually every robot form factor.
Auto-Engineering puts the GRID monorepo to work and makes it compound. As physical AI advances, GRID absorbs each new model, algorithm, simulator, and technique, turning every advance into one more capability its agents can build with. Every problem Auto-Engineering solves expands what it can solve next.
Robotics harnesses that unlock Auto-Engineering:
We identify four crucial harnesses to enable autoengineering for robotics as highlighted in above figure. Our harnesses were then put to test using a real world laboratory use case: pick up a test tube, pass it between two robot arms, and pour into a beaker. We then kept increasing task complexity and had autoengineering leverage our designed harnesses for solving the tasks.
Robot Ingestion Harness
Our first harness focuses on understanding robot morphology and its operational context: robot’s joints, grippers, cameras, workspace, control interfaces, and calibration. It converts that information into concrete artifacts that GRID can use across simulation and real-world deployment. Those artefacts also provide the basis for transferring existing skills between robots with compatible morphologies, rather than rebuilding each behavior from scratch. GRID’s common abstractions for hardware, control, and deployment give agents a consistent interface across otherwise very different robotic systems.
For our initial laboratory task: we focused on picking a test tube from a rack, and pour into a beaker for a dual-arm setup with two Flexiv Rizon arms. We then introduced a new requirement: transfer the behavior to a UR5e. The harness integrates both robots and the generated descriptions of their hardware and control interfaces become concrete artefacts. GRID’s agents then used that knowledge to adapt the skill to the UR5E’s setup.
These artifacts also help us build preflight checks that can investigate the deviation from assumed operational contexts such as novel grippers, altered sensor mounting, calibration drift, or an unexpected control frequency. GRID can investigate those changes, update its understanding of the robot, and test the correction.
World Experience Harness
The World Experience Harness makes simulation itself an engineering variable. Rather than being constrained by the capabilities of a single simulator, GRID can select, combine, and extend physics solvers, renderers, sensor models, and simulation backends around the task. In effect, GRID is now able to assemble simulations that did not previously exist.
For our pouring task, the world needed to model both rigid-body motion and the liquid inside the test tube. GRID constructed a hybrid simulation by coupling a DFSPH fluid solver implemented in NVIDIA Warp with MuJoCo rigid-body dynamics, bringing robot motion, vessel contact, and liquid transfer into the same environment. The harness then turned that world into experience through synthetic demonstrations, randomized conditions, and experiments targeted at failures. Highly extensible, physically-grounded simulation gives robotics harnesses what tests give coding agents: a scalable way to verify outcomes, expose unsafe behavior, and improve before returning to hardware.
The same principle extends to very different problems. GRID can construct a warehouse world that combines an industrial arm with a simulated PLC controlling a conveyor; bring a Gaussian-splat reconstruction of a real environment into simulation for testing navigation; or integrate additional physics when a task requires deformable objects such as cloth.
Skill Creation Harness
The Skill Creation Harness gives agents several ways to build and test robot skills: composing existing models and controllers, generating synthetic demonstrations and training policies, collecting human demonstrations, fine-tuning VLAs, or learning from human video. It connects data generation, training, inference, and evaluation so that GRID can choose an approach based on the task and revise it based on results. Not every task requires a new model.
- Modular harness: For the initial pouring task, GRID composed capabilities that were already available: object segmentation for locating the test tube and beaker, grasp generation, and collision-avoidant motion planning. The skill operated on objects, their spatial relationships, and end-effector motion rather than a fixed sequence of robot joint commands. As a result, the test tube and beaker could move within the workspace without requiring the behavior to be rebuilt for each configuration. GRID checked the composed skill in physics-based simulation for reachability and collisions before hardware evaluation, retaining the component configurations and test results so the deployed behavior could be traced to what had been verified.
The same skill could also be transferred across compatible robots through GRID's common robot abstractions. In this case, GRID did not need to train anything new, and built the entire skill autonomously.
- Policy harness: We then increased the task complexity by moving the beaker during the pour. The robot now had to track the beaker and keep the test tube aligned with it while handling the liquid. GRID generated expert demonstrations in simulation and trained a reactive policy using behavior cloning and DAgger, entirely from synthetic data. Since cross-robot transfer was also a requirement, GRID chose a state-based representation with end-effector actions rather than learning a policy tied directly to the joints of a particular robot and evaluated it on the UR5e and Flexiv.
- Demonstration harness: We next asked GRID to pick a thin stirrer from a stand and stir inside a beaker. Stirring requires sustained contact and coordinated motion that are difficult to express using the available motion primitives. GRID determined that human demonstrations were needed and used its teleoperation and training pipelines to request data collected with GELLO arms. It converted those demonstrations into a dataset, fine-tuned a visuomotor policy, and deployed the resulting policy for evaluation.
The collection process can itself respond to deployment feedback. If a policy repeatedly struggles with a particular grasp or object configuration, GRID can request demonstrations targeted at that failure rather than simply collecting more of the same data. A subsequent collection round might focus on a specific recovery, starting position, or grasp approach that the current policy does not handle well.
- Video-to-Sim harness: Finally, we considered an orthogonal task: swirling liquid in a flask. Swirling requires rapid, coordinated circular motion that is difficult to reproduce through conventional robot teleoperation. We therefore gave GRID an additional constraint: no robot demonstrations, and only a single phone video of a person performing the task.
GRID selected methods for hand-pose estimation and 3D object segmentation, tracked the hand and flask, recovered their motion in 3D, and reconstructed the interaction in simulation. It then varied the reconstructed scene, checked the resulting motions against the observed behavior, and generated synthetic demonstrations to train a behavior-cloning policy. Each generated demonstration remained traceable to the source video and observable in simulation, with physics-based checks grounding what the policy learned.
One human video became a world of robot experience.
Deployment & Evaluation Harness
A skill that works in simulation still has to work reliably on the physical robot. The Deployment & Evaluation Harness provides feedback throughout this process, from simulation and preflight checks to evaluation on hardware. GRID uses this evidence not just to score a skill, but to determine what needs to change next.
Before execution, preflight checks invoke system identification to validate assumptions about the robot—its kinematics, coordinate frames, calibration, sensors, gripper, and control interfaces. In one case, GRID found that the assumed robot model disagreed with real controller measurements by 143 mm and 6.4°. GRID was able to correct the model and frame iteratively and reduced that discrepancy to 5.7 mm and 0.68°.
Hardware evaluation then exposes the remaining sim-to-real gaps. Some required relatively small changes: segmentation prompts that worked in simulation had to be tuned to reliably identify the real test tubes and beakers. Others forced GRID to build new capabilities. For reactive pouring, the state-based policy needed the beaker position at real-time control rates, which the existing segmentation pipeline could not provide. GRID generated supervision from its segmentation pipeline, fine-tuned a lightweight detector/tracker for the real scene, and inserted it back into the skill to close the perception loop in real time. This is an important part of Auto-Engineering: deployment can reveal a missing component, and GRID can create that component as part of getting the skill to work.
The feedback can reach much further down the stack. When reducing the robot's configured speed produced little change, the deployment harness found that commands were being sent at roughly 500 Hz rather than the intended 30 Hz. It corrected the command pacing and verified the repair at 29.7 Hz. In another case, when inaccurate scene geometry began causing problems during motion planning, GRID traced the issue back to depth estimation. It evaluated multiple depth models on the same camera capture and selected one that reduced the reconstructed tabletop’s plane error from 28.0 mm to 1.2 mm. The more accurate geometry then exposed collisions that the previous depth map had hidden.
From a fresh robot to the first working skill, this complete process took about four hours on the Flexiv: roughly 20 minutes for robot ingestion, 10 minutes for the initial simulation, and the remaining time for skill creation, evaluation, deployment, preflight checks and refining based on deployment feedback.
Those four hours changed the starting point. GRID retained the robot integration, simulation, perception adaptations, control fixes, and deployment procedures for the next task. Later skills on the same Flexiv setup were deployed in as little as 10–15 minutes.
This is the role of Deployment & Evaluation in Auto-Engineering: a deployment produces more than a working skill. The failures, experiments, and validated fixes become engineering knowledge that GRID can use the next time.
Intelligence that compounds
The output of Auto-Engineering is not just a working robot behavior. Each engineering loop leaves behind things that GRID can use again:
- Embodiment knowledge: robot descriptions, calibrations, coordinate frames, control characteristics, and integration tests.
- Skills: reusable behaviors and task components that can transfer across objects, scenes, and compatible robots.
- Models and capabilities: perception models, trackers, simulation components, training recipes, and other functionality created or adapted while solving a task.
- Failure and repair knowledge: what failed, how GRID tested it, which approaches were rejected, and which repairs survived evaluation.
These functions both arise from and feed back into the harnesses. A deployment failure can update GRID's understanding of an embodiment, trigger a new simulation experiment, lead to a perception model being adapted, or change how a skill is constructed. Once the result is validated, it becomes part of the engineering available to the next loop.
We saw this within our lab experiments. Perception code developed for one deployment was reused on another robot. A control-timing repair became part of the shared motion layer. Failed hypotheses were retained rather than disappearing with the debugging session. The next task therefore did not start from the same place as the first.
We started with a lab bench. GRID came away with new skills, models, simulation capabilities, robot knowledge, and deployment experience it can use again. As GRID encounters more robots, tasks, objects, and environments, that foundation continues to grow. That is how robotics harnesses enable Auto-Engineering: they give agents the means to turn intelligence into reliable behavior, and make every deployment strengthen the system that engineers the next. We are applying this paradigm across various form factors and applications: industrial arms, dexterous hands, navigation robots, as well as humanoids.
GRID is designed to make that engineering compound, which in turn lets us build useful robots faster.