Peking University, Qiyuan Robot launch HIL-UMI to cut real-robot reliance in embodied post-training

In embodied AI, post-training has relied on real-robot data, often through DAgger-style human intervention: an operator watches a robot arm, takes over when it is about to fail, records a correct trajectory and then resets the scene. The process is slow and costly, and scaling it to many robots requires hardware, space and labor. Handheld UMI devices can collect data cheaply and at scale, but conventional UMI collection captures only positive demonstrations and cannot supply the targeted corrections needed for long-tail failures, according to the report.

HIL-UMI adds a model to the UMI collection loop. An operator holds a UMI gripper and performs tasks such as picking up toys or folding towels while cameras record the scene. In the background, the model receives the same images and task instruction and predicts the next 20 actions, but it does not control a robot. The system compares the human’s actual actions with the model’s predictions. When the two are close, it treats the model as already competent; when the predicted trajectory diverges sharply, it prompts the operator to demonstrate more data from that point. No robot arm is needed, and the robot does not have to fail physically first.

The system uses an action-divergence detector that samples multiple action chunks and computes an energy score. A concentrated distribution suggests the model is confident; a dispersed distribution that also departs from human action is treated as an out-of-distribution state and triggers targeted collection. A second component, an advantage estimator, evaluates whether an action actually advances the task, distinguishing useful corrections from idle motion. After each round of targeted data collection, the model is updated; the next round’s disagreements move to harder edge cases, creating a data flywheel.

In tests on four real-robot tasks—folding towels, organizing a tabletop, stacking blocks and stamping—the team compared standard supervised fine-tuning with HIL-UMI under the same data-frame budget. Regular SFT improved average task progress score from 44.3 to 58.0, while HIL-UMI reached 94.8, according to the report. On the precision-heavy block-stacking and stamping tasks, success rate reached 100%. In the long-horizon tabletop-organizing task, real-machine HG-DAgger took 412.99 milliseconds to collect one effective frame, while HIL-UMI took 73.40 milliseconds, a 5.63-fold improvement in collection efficiency.

The report positions HIL-UMI between existing routes: real-robot teleoperation and DAgger offer high-fidelity data but are expensive and hard to scale; simulation and reinforcement learning are cheap but face sim-to-real gaps, especially with deformable objects, friction and lighting; massive video pretraining provides common-sense perception but lacks precise action data; traditional UMI is cheap but positive-only. HIL-UMI borrows UMI’s low cost and DAgger’s targeted correction, according to the team.

Dong Hao, Peking University associate professor and Qiyuan Robot chief scientist, told Leiphone that pretraining raises the lower bound while the upper bound depends on post-training. He compared the shift to AlphaGo, which began with imitation learning from human games and then advanced through large-scale self-play in post-training. In his view, pretraining and post-training are a stage-based succession rather than a simple ratio, and future competition will hinge on scalable post-training that can build correction networks at low marginal cost and high speed.