IROS 2026 Talk Maps Embodied AI Path From Ground Robots to Aerial Construction Teams

Alonso-Mora leads the Autonomous Multi-Robots Laboratory at TU Delft and has long studied how multiple autonomous agents plan and coordinate in dynamic environments. His earlier research on shared mobility scheduling led to The Routing Company, which Leiphone described as one of the few on-demand mobility scheduling technology companies to achieve scaled industrial deployment.

In ground mobile manipulation, the team worked on a supermarket picking scenario. A robot equipped with a mechanical arm moves between shelves and reaches out to grasp products. For whole-body control, it uses Geometric Fabrics, a set of control methods that can define behaviors such as collision avoidance and target reaching, then combine them to control both the arm and the mobile base at high frequency. A behavior tree handles higher-level task flow, including when to grasp, which object to grasp, and whether to retry after failure. Because reactive control can become trapped in local optima, such as two robots at a table blocking each other in a deadlock, the team combined the low-level Geometric Fabrics controller with a global quadratic programming system. The system forward-simulates the whole-body controller to predict possible conflicts, then re-optimizes actions such as the approach direction, grasp pose, and final placement.

For more complex tasks, the team worked on task and motion planning, which combines discrete decisions about task order with continuous decisions about how each step should be executed. In one example, a robot had to pick up a block from one table and place it on another, with a ramp-like obstacle between the tables. The system sampled different grasp points, placement points, and actions, forward-simulated them, and used cross-entropy optimization to improve the plan. Physical simulation was carried out in Isaac Lab. The resulting plan was unexpected: instead of carrying the block around the ramp, the robot sent the block over the ramp, because the simulation and optimization found that this was faster. The behavior was not a pre-designed rule. The method was later executed on a real robot. It worked well but was computationally expensive, so the team is now exploring learning policies from demonstrations to reduce the need for extensive simulation-based search.

The team also extended mobile manipulation toward multiple robots. In one experiment, two mechanical arms were trained separately to pick up and place blocks, then combined through a joint optimization objective to complete a shared task. The first robot picked up the block and handed it to the second, which carried it farther. A practical problem in multi-robot learning is that collaboration demonstrations are difficult to collect, because collecting them requires controlling multiple robots at the same time. The team hopes to reuse existing single-robot demonstrations to enable multi-robot collaboration. The talk identified construction as a more challenging direction, where humans can cooperate to carry large objects and perform complex assembly, while robots remain far from that capability.

Aerial robots are the focus of the laboratory's current work. The first type is a drone with a mechanical arm. Unlike ground robots, aerial platforms have no fixed support, so arm movement or contact forces can affect the drone's own stability. The team's platform, called the Differential Shoulder Aerial Manipulation Platform, or DSAM, uses layered control: an outer reinforcement learning policy generates manipulation commands, while an inner adaptive controller stabilizes the system. The drone can push a box, write on a whiteboard with a pen, and tighten bolts.

The second type of aerial robot is designed for cooperative transport. In one demonstration, three drones used ropes to lift an object together. Initially, the team used centralized nonlinear model predictive control, with a central controller computing control inputs for all drones to track trajectories, avoid obstacles, and resist external disturbances. If the transported object was hit, the system could readjust and continue along the target trajectory. The team then tried to remove the central control unit. It tested multi-agent reinforcement learning and imitation learning. In the reinforcement learning approach, training took place in simulation with an actor network and a critic network. During training, the critic could use full state information from the simulation, but during execution each drone ran only its own actor. A high-speed INDI controller below the neural network handled flight stability. In an figure-eight trajectory tracking experiment, the distributed policy performed close to the centralized nonlinear model predictive control method. The team also tested extreme cases, including adding another drone, making one drone fail and move in the wrong direction, or even drop, and the policy showed some robustness. Limitations remain: the distributed learning method has not yet incorporated obstacles, training with obstacles would be more complex and take longer, and the system still relies on motion capture equipment.

The next step is a new four-year project called HARPA, short for Heterogeneous Aerial Robotic Teams for Prefabricated Building Assembly. The project runs from 2026 to 2030. It will use both types of aerial robots. Some drones will transport building materials to construction positions, while drones with mechanical arms will handle assembly, such as connecting building modules and installing bolts. The project will also advance onboard perception so robots can rely more on their own sensors.