NIO Paper Proposes MM-Future Multi-Mode World-Action Model for Autonomous Driving

Ren is a chair professor at USTC and, according to USTC’s website, director of its Institute for Artificial General Intelligence; he remains head of NIO’s smart driving business. QbitAI noted that his representative academic work was concentrated in 2014–2016, including Faster R-CNN and ResNet, and that MM-Future marks his return as corresponding author on an autonomous-driving paper. The paper lists NIO as its first institution, with many authors from NIO.

The paper addresses a scenario in which a driving system must handle more than one trajectory or one deterministic future. Existing world-action models, QbitAI reported, generally fall into two categories. Cascaded systems generate candidate trajectories and then predict futures, or predict futures and then plan; information moves in one direction, so a future result cannot revise an earlier decision. Joint systems let scene and action affect each other, but usually generate only one scene-action result. MM-Future generates multiple paired scene-action hypotheses, with each trajectory and its corresponding future scene evolving together before selection.

To create diversity from single-outcome driving data, the model uses Gaussian Mixture Noise on the action side based on the training trajectory distribution and independent noise on the scene side. Action and scene initial states are paired. During training, a Best-of-Many supervision scheme first finds the candidate closest to the real trajectory, then uses the same winner index to supervise both action and scene streams. This is intended to prevent candidates from being pulled toward similar results and to keep a trajectory paired with its predicted future.

For computation, MM-Tokens compress multi-camera, multi-frame visual features into a compact planning-oriented representation. The tokens do not reconstruct RGB images or recover a full BEV; the limited budget retains information related to future evolution and planning. Attention visualizations cited by QbitAI focus on road structure, intersections, front vehicles and surrounding traffic participants. As a result, the model does not need to generate full high-definition future video for dozens of candidates, keeping an internal set of compact future scene representations for planning.

MM-Future uses a modality-aware Transformer to process action and scene streams. Shared attention handles interaction between the two types of information, while modality-specific branches preserve their statistics. Candidates share model parameters but do not exchange tokens across modes, so trajectories update as paired scenes change and scene predictions adjust to ego actions. A Future-Conditioned Proposal Scorer then scores each trajectory using history and its own paired predicted future, without borrowing other modes’ scene results. Stop-gradient is applied to trajectory and future tokens. Inference has four steps: encode history, generate multiple scene-action results, score candidates with history and paired future, and output the highest-scoring trajectory.

On NAVSIM-v1 navtest with trainval data, MM-Future reached 94.0 PDMS, compared with 90.7 for the WAM method DriveFuture and 93.7 for the E2E method DrivoR trainval in the paper’s table. On NAVSIM-v2 it reached 91.5 EPDMS, above UniTeD at 90.1 and DriveFuture at 89.9. In ablation tests, action-only single-mode planning scored 84.1 PDMS; increasing action candidates to 32 raised it to 92.3. Adding joint scene-action generation produced 85.1 in single mode and 92.9 with 32 modes. Letting the scorer read each trajectory’s predicted future raised PDMS to 93.3. The paper attributes the clearest scorer gains to TTC, suggesting candidate-specific futures help exclude trajectories with higher interaction risk.

Multi-mode training also converged faster in the reported setup: 16-mode and 32-mode configurations reached a 0.80 validation PDM score in about 3.8k steps, while the single-mode version required 17.5k steps. The main model samples 64 scene-action candidates, with an end-to-end forward latency of about 233ms on a single NVIDIA H800 with batch size 1 and bf16, QbitAI reported. Latency for 16 candidates was close to the single-mode approach, while 32 candidates increased it clearly. In closed-loop tests without HUGSIM fine-tuning, MM-Future scored 32.3 average HD-Score over 436 scenarios, above Latent-WAM at 28.9 and UniAD at 28.6. Its average Route Completion was 44.5, slightly below Latent-WAM’s 45.9, and under Extreme difficulty its HD-Score was 8.6 versus 18.1 for Latent-WAM.

The paper also notes interpretability limits: MM-Tokens are implicit scene representations, and it remains difficult to observe directly what planning information the model retains. The authors plan to add an auxiliary perception module to visualize planning-related scene structure and aid fault diagnosis. The work remains at the stage of public datasets and simulation validation, with computation cost, implicit representations and stability in extreme scenarios still requiring further verification.

The research team is mainly from NIO’s intelligent driving foundation model pre-research team, according to QbitAI. First author Shuai Liu is affiliated with NIO and the School of Computer Science at Sun Yat-sen University; he previously published GaussianFusion as first author in 2025, which was accepted by NeurIPS 2025. Hechangle Gong is affiliated with NIO and Beihang University. Hao Jiang, Runlin He, Junxiang Zhan and Sheng Yang are listed with NIO, and Kai Huang is from Sun Yat-sen University. Ren Shaoqing is corresponding author, affiliated with NIO and the USTC AGI Research Institute.

Within NIO’s technology line, 2024 brought NIO WorldModel and NADArch 2.0, shifting the architecture further toward a world-model-driven end-to-end system; 2025 added full closed-loop reinforcement learning, and the latest NWM began large-scale rollout in early 2026. NIO disclosed that by September this year, the system based on the world model and full closed-loop reinforcement learning had been deployed to more than 950,000 vehicles, QbitAI reported.

QbitAI reported that MM-Future’s current value is best understood at the method level. The role of an autonomous-driving world model is moving from future prediction toward planning, requiring the model not only to estimate how a scene may change but also to compare different scene outcomes under different actions. Previous multi-mode planning mainly asked which trajectories could be taken; MM-Future extends the question to how the environment may change after each trajectory. The paper’s supported conclusion is that covering multiple possible outcomes while keeping bidirectional interaction between scene and action yields stable metric gains.