Fudan’s Jia Xiaosong Says Autonomous Driving’s Bottleneck Is Not Model Architecture

Jia began his PhD at Shanghai Jiao Tong University in 2021. Late that year, his advisor Yan Junchi called him and asked what he wanted to study. Jia planned to continue trajectory prediction, a field he entered as an undergraduate through a summer research program at UC Berkeley in 2019 in Masayoshi Tomizuka’s team, where he worked on vehicle interaction and trajectory prediction. Yan advised him to switch, arguing that prediction sits between perception and decision-making and is constrained by both. Jia said he had also seen limited room for improvement in trajectory prediction. Under Yan and Li Hongyang, founder of OpenDriveLab at Shanghai AI Laboratory, he began research on end-to-end autonomous driving.

In 2022, Jia and collaborators developed TCP, which kept separate trajectory and control branches and found that fusing them performed well. He said such intermediate tasks remain important because they provide supervision signals in a setting where images and point clouds are plentiful but driving trajectories are sparse, though he no longer treats them as true interpretability. At RSS 2026, he published GuidedVLA to discuss the issue from an embodied-intelligence perspective. He also said rules remain necessary for safety, insurance and regulation, describing them as a harness around neural planners.

Think2Drive, presented at ECCV, let a model simulate interactions in a low-dimensional latent space and cut the cost of policy training. Jia said the work trained a policy on a single NVIDIA A6000 GPU in about three days to complete complex CARLA Leaderboard 2.0 driving tasks. After that, he moved into closed-loop evaluation and interactive generative simulation, and in 2023 began introducing world models into autonomous-driving reinforcement learning.

His doubts about imitation learning came from closed-loop simulation. In ThinkTwice, he said, the team collected what it regarded as nearly unlimited data for CARLA 1.0 and trained larger models with support from Yan and Li, using a 100TB hard drive and substantial compute. But scaling data and model size did not take driving from acceptable to near-perfect. Jia concluded that the problem lay in imitation learning’s objective, which rewards similarity to human demonstrations rather than the quality of a decision. He cited traffic lights: a model can learn that it should move when the car ahead moves, a correlation that lowers training loss on most samples without learning the actual rule. Mean squared error can also average different but valid human actions into an unreasonable one, and imitation data rarely teaches a model how to recover after its own mistakes.

He then built DriveAdapter, which connected an existing reinforcement-learning planner to perception modules and was presented as an ICCV 23 Oral. Jia described it as an early two-stage end-to-end approach: train separately, then combine. He said reinforcement learning from scratch converges slowly and, while it can handle dozens of CARLA scenarios, he was not confident it could master thousands of real-world scenarios without imitation-learning pretraining. His current view is to imitate first and then reinforce: imitation takes a model from zero to 90, and reinforcement learning addresses the remaining 10. The harder problem, he said, is scaling reinforcement learning so one general model can improve across many tasks.

Jia now works on dexterous hands and ego data at Fudan. He said the problems autonomous driving encountered in recent years are reappearing in embodied intelligence in different forms.

Editor's Summary

Jia Xiaosong’s account shows how autonomous-driving research moved from modular prediction to end-to-end learning, reinforcement learning and world models, while exposing the limits of imitation learning and the cost of scaling reinforcement learning. His work, including Think2Drive and DriveAdapter, illustrates that architecture changes alone do not settle the field’s training and safety problems. The same issues are now resurfacing in embodied intelligence.