PlanWAM: Optimizing Future Scenario Planning for Autonomous Driving

Paper: PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving

Authors: Jinchang Xu, Hongda Yu, Fengwei Dong, Wenhui Huang, Xi Wei, Yongzhi Liu, Sunan Zhang, Jirao Wang, Chen Lv, Bingbing Li, Guodong Yin, Weichao Zhuang

Executive Summary

This research introduces PlanWAM, a novel approach in the field of autonomous driving that enhances trajectory planning by optimizing how future scenarios are represented. Current methods in autonomous driving primarily focus on predicting future events and utilizing these predictions for decision-making. However, they often overlook the question of which types of future representations are most beneficial for effective planning.

PlanWAM addresses this gap by tailoring the future-state representation specifically to the needs of the planning task. It achieves this by employing a unique framework that integrates historical data with trajectory planning objectives. The process begins with a Temporal Register Pyramid, which efficiently compresses multi-frame historical information while prioritizing the most relevant data for future reasoning. This method ensures that the representation captures only the necessary information, improving the model's efficiency and effectiveness in predicting future scenarios.

Importantly, PlanWAM includes a privileged future posterior branch that accesses actual future frames. This allows the model to refine its future latent representation based on real trajectory-planning goals. Consequently, the model gains a more nuanced understanding of potential future states, which aids in generating and selecting trajectories.

The effectiveness of PlanWAM is evaluated through benchmarks on two simulation environments, NAVSIM-v1/v2 and HUGSIM. It achieves notable scores: 93.8 in PDMS and 90.9 in EPDMS on NAVSIM, and 38.7 in HD-Score on HUGSIM, indicating superior planning performance in both open-loop and closed-loop settings. These results suggest that by focusing on planning-shaped representations, PlanWAM not only enhances the foresight capabilities of world-action models but also demonstrates a viable path for improving autonomous vehicle navigation systems.

The implications of this research are significant for the development of autonomous driving technologies. By refining how future states are represented, companies involved in autonomous vehicle development could potentially improve the precision and reliability of their trajectory planning systems. This could lead to safer and more efficient navigation, benefiting various stakeholders in the automotive and transportation sectors. The findings emphasize the importance of integrating planning needs into predictive models, which could open new avenues for research and application in autonomous driving.

Academic Abstract

World models in end-to-end autonomous driving predict future scene evolution to provide foresight for trajectory planning. Existing methods mainly study how to predict the future and how to use it, but less often ask which future representation is actually most useful for planning. To this end, we propose PlanWAM, a Planning-Shaped World Action Model. The key idea is to let the planning task shape the future-state representation, so that it retains the information most useful for planning. A latent world model then predicts this planning-shaped future latent representation from historical observations and uses it for planning, enabling foresighted planning. Specifically, we first use a Temporal Register Pyramid to compress multi-frame historical information in a recency-aware manner, learning a compact history representation oriented toward future reasoning and planning. We then introduce a privileged future posterior branch that observes ground-truth future frames, and shape its future latent representation with trajectory-planning objectives to obtain a planning-shaped future latent representation. Hindsight-to-Foresight Distillation trains a prior branch that depends only on history to predict this future latent representation. The predicted future latent representation serves as planning context and guides trajectory generation and selection. PlanWAM achieves 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and reaches 38.7 HD-Score on closed-loop HUGSIM in a zero-shot setting, demonstrating leading planning performance across both open-loop and closed-loop evaluations. Extensive experiments further demonstrate that planning-shaped future representations provide an effective and deployable form of foresight for world-action models.