It’s day four of Robot Week! You can click here to get 25% off an annual subscription.
On May 28, the startup Shift announced that it would clean any New York City apartment for free. In a launch video, cheerful young men scrubbed toilets, vacuumed floors, and wiped down counters.
The catch? Cleaners wore baseball caps with cameras mounted under the brims. The company planned to record the workers’ actions and sell the data to robotics companies.
While the whole deal might have been a gimmick — the scheduling website notes in an FAQ that the offer is only available for a “limited time” — it’s still a perfect encapsulation of one of the most important trends in robotics today.
Early LLMs were famously trained to “predict the next word” across billions of tokens of text scraped from the Internet. Most roboticists expect we’ll need something similar to train general-purpose robots: an Internet-scale database of everyday tasks that robots can learn from.
But right now, humanity doesn’t have anything like that. The largest openly available dataset of robots performing tasks, ABC-130K, only has 3,500 hours of task demonstrations.
Over the last few months, I’ve talked to dozens of founders, engineers, and robotics researchers about the need for demonstration data and the ways people are trying to get more of it. I visited a robotics lab at the University of Pennsylvania to try my hand at collecting robot data. During my spring trip to China, I watched men wearing virtual reality headsets puppet humanoid robots to open fridges, sweep trash, and move pillows around.
When I attended the Actuate conference in San Francisco in August, I was surprised by how many people there were working at data-collections startups.
A plethora of startups like Shift are trying to solve the data shortage by recording the actions of humans and converting the videos into training data for robots. Other companies are hiring humans to directly operate robots in labs, factories, and even people’s homes. Still others are hiring humans to perform everyday tasks while wearing gloves or exoskeletons that force the human to move in a robot-like way and capture rich data on the worker’s actions. Some large data-collection companies like Scale AI are experimenting with all of these strategies.
The ultimate goal is to develop robots that are good enough to operate (mostly) autonomously in the real world. Once that happens, robots could generate additional training data while doing useful work. This could lead to a flywheel where the companies with the best robots are able to generate the most high-quality data, allowing them to improve their robots even more.
But Deepak Pathak, the CEO of robotics startup Skild, told me that there’s a “chicken-and-egg problem” here. In order to generate high-quality data from deployments, robots need to be able to do some amount of useful work. And getting there will probably take a fair amount of data. So companies first need to figure out a scalable way to get robotics data without deploying robots commercially. The first company to figure this out could have a big advantage.
Getting the computer to make the data for you
Before we explore the strategies companies use to generate real-world training data, it’s worth asking why we need real-world data at all. Nearly a decade ago, Google DeepMind trained an AI to play Go entirely by self-play. After playing millions of games against itself, the model became better at Go than the top humans.
Could we do something similar for robots? Instead of training physical robots in the real world, maybe we could have virtual robots “teach themselves” to perform tasks through trial and error in a simulated environment. This approach actually does work for certain robotics tasks.
In March 2025, the humanoid robotics company Figure posted a high-level description of how it trains its robots to walk. Figure programmed a digital twin of its Figure 02 robot in a physics simulator and had that virtual robot try to walk over and over for millions of attempts. Each time, the robot received programmatic feedback — in a process called reinforcement learning — until the robot could walk in simulation. When Figure installed the resulting model on a physical robot, it could walk in the real world too.
When this process works, it’s the ideal way to train a robot.
It can be very fast: Figure said it was able to obtain “years of simulated demonstrations in a few hours.” This method can also result in a very robust model: after training for the equivalent of 1,000 years in a simulator, the foundation model company Skild produced a model that could control a quadruped robot even when engineers sawed its legs in half.
Basically every company today making a humanoid robot uses reinforcement learning in a simulated environment to teach it how to walk.1 Unfortunately, while this approach works well for locomotion tasks like walking and dancing, it doesn’t work as well for manipulation tasks, which involve more complex interactions with the environment.
Imagine trying to train a robot to hammer a nail. If a robot starts out acting entirely at random, it might go through millions of iterations without a single success. Reinforcement learning works by “rewarding” the model when it succeeds, but if the model never succeeds, there’s nothing to reinforce.
Developers can help the virtual robot by giving it fine-grained feedback that acts as a trail of breadcrumbs along the path to success. The robot might earn points for touching the hammer, more points for picking it up, still more for touching the nail with the hammer, and so forth. But this technique, known as “reward shaping,” is labor-intensive, doesn’t transfer well between tasks, and still may not produce good results.
In 2017, when prominent researchers tried to use reinforcement learning to teach a robot to hammer a nail in simulation, they couldn’t get it to work with just a “sparse” reward that judged whether the robot succeeded at the overall task. With help from shaped rewards, it took 50 hours of training for the robot model to learn — but the robot’s technique was still awkward:
However, if the researchers provided 25 demonstrations of a human completing the task, the robot model learned how to do the task in about six hours — almost 10 times faster. And the robot wound up with better hammering technique.
While this paper is almost a decade old now, the basic observation is still true: in order to learn from trial and error, it’s helpful for the model to start with a certain level of basic competence so it succeeds at least some of the time. And one of the best ways to achieve basic competence is to have it first learn from human examples.
The sim-to-real gap
There’s another problem with trying to train a robot entirely in simulation: many aspects of the world are so complex that we don’t know how to simulate them with enough fidelity.
Take the hammer example again: one of the reasons the model learned to use an awkward grip that probably wouldn’t work in real life is that the simulator couldn’t model friction perfectly. This discrepancy between simulation and the real world — the sim-to-real gap — is one of the central challenges developers face in training robots in simulation.
Some research groups are optimistic about the sim-to-real gap. At the GTC conference in March, I talked to Ranjay Krishna, who recently co-supervised a research project at the Allen Institute for AI (Ai2). “Our bet was that the sim-to-real gap is something we can overcome with large amounts of diversity in simulation,” Krishna told me.
The idea is to use large-scale randomization to make robotic models more robust. Randomization is already a standard technique — when teaching a robot to walk, companies will simulate thousands of different terrains for the robot to walk over. The Ai2 research group scaled it up for manipulation tasks: the researchers generated 5,704 hours of programmatically generated simulation trajectories across 94,200 distinct simulated environments. They also randomized other parts of the scene, like what cameras the robot had access to.
The results were promising, albeit somewhat narrow. When deployed on a real-world robot, the model was able to complete several tasks that involved rigid objects, like putting an apple on a plate. However, the researchers did not attempt more difficult tasks. As they explained in their paper:
We focus on rigid body and articulated object manipulation — tasks where modern simulators provide sufficient fidelity for transfer. Extending to contact-rich manipulation (e.g., insertion, peg-in-hole), deformable objects (cloth, rope, food), or tasks requiring accurate fluid or granular dynamics remains an open challenge. We believe that coupled with advances in physics-based and generative world model simulators, our recipe of massive-scale procedural generation may extend to these more challenging tasks requiring contact-rich dexterity and deformables.
Deepak Pathak, the CEO of Skild, has a similar view. When we talked in August, he argued that if a model is trained to adapt to a large enough variety of simulated environments and robotic embodiments, then it will be able to adapt to varied real-world situations as well. He hinted that future Skild releases would demonstrate such a capability. Later in the month, Skild released S1, which showed an impressive ability to pick up new tasks from humans.
Robot Week special: Click here to get 25% off an annual subscription.
Collecting data in the robot embodiment
However, most of the experts I talked with don’t share Krishna and Pathak’s optimism about simulation.