Video2DoorTraversal: New Framework Lets Robots Learn Door Opening from a Single Video

In the main experiment, for each of five target doors, a separate RGB video was collected. The specified wheeled-legged platform was tested 35 times per door, for a total of 175 trials, of which 169 succeeded, giving an average success rate of 96.57%. On three structurally similar but unseen doors, the policy achieved a zero-shot average success rate of 80.95% without additional training or trajectory generation. On average, the whole process from approaching to completing traversal took about 13 seconds, with visual processing and policy inference performed onboard.

The paper notes that real building doors vary in panel size, hinge direction, handle type, installation position, and opening resistance. Existing approaches often rely on pre-built door assets, extra scanning, manual task design, or real-robot adaptation. Video2DoorTraversal attempts to reduce some of this upfront work.

The first step converts a real door into a simulation-ready DoorTwin. From a single RGB video, the system recovers geometric information with real-world scale and camera motion, identifies the door panel, handle, and hinge relationships, and generates a digital asset with joints, collision geometry, and appearance texture. It then checks silhouette, proportions, handle type, hinge direction, and part positions from reference views, iteratively correcting the result. The digital door must be both visually close to the real object and physically graspable, rotatable, and pushable, so that simulation practice can transfer to the real robot.

After creating DoorTwin, the simulation-based closed-loop agent organizes the task into parameterized skills such as approaching, grasping, turning the handle, pushing the door, and traversing. It repeatedly executes the skills in the simulator. When a trajectory fails, the system analyzes the cause from collisions, contacts, handle rotation, and door state, and continues searching in neighboring parameters. Only trajectories that pass task, collision, and kinematics verification are kept as training data. This process does not rely on teleoperation collection for each door. To improve sim-to-real adaptation, training also randomizes initial poses, joint friction and damping, opening resistance, camera extrinsics, and adds depth noise. Only trajectories that succeed under disturbances are retained for policy training.

There are two distinct input stages: the initial RGB video is used to generate DoorTwin, while during real-robot execution, "ArticuACT" uses dual-view depth images from the forward-facing and wrist cameras plus robot states to coordinate commands for the base, arm, and gripper. The model is also trained to predict contact, handle rotation, and door opening progress to learn interaction features at different stages.

The 96.57% success rate reflects the average real-robot result on five specific push doors under the paper's settings, showing end-to-end verification on the specified objects, wheeled-legged platform, and test setup. It does not represent a general-purpose door-opening product or a company-level engineering closed loop. The 80.95% result corresponds to zero-shot tests on structurally similar unseen doors. The policy trained on simulation data from one reconstructed door succeeded on new doors of similar structure without additional training or trajectory regeneration. This gives preliminary evidence of cross-door transfer, but with clear boundaries: the current validation objects are structurally similar push doors; pull doors, more handle mechanisms, and richer door structures are listed as future work. For industrial deployment, valuable metrics include new-scenario adaptation cycles, on-site trial counts, manual intervention costs, and whether simulation assets and training data can be reused across more facilities, scenarios, and robots.

Newwa Robotics defines embodied intelligence as continuous passage and task completion in open human environments, describing it as the intersection of embodied intelligence and autonomous driving. Its World Traversal Model (WTM) aims to organize spatial understanding, task judgment, navigation, and embodied manipulation in one intelligent base. In its company-level architecture, WorldWeaver is positioned as the organization layer for real-scene data and 3D assets, while SimWeaver aims to provide physics simulation capabilities and support WTM training and evaluation. The paper does not indicate that Video2DoorTraversal has formed a unified engineering loop with WorldWeaver, SimWeaver, or WTM, though they share a technical path: letting real scenes enter a digital environment, letting robots accumulate experience in simulation, and then returning to real tasks for verification. Doors are only common passage facilities in open human environments; the future value of this research will depend on whether similar methods can cover more facilities and tasks and continue to reduce the cost of adapting and validating new scenarios.