AdaWM: Adaptive World Model based Planning for Autonomous Driving

Understand

World model based reinforcement learning (RL) has emerged as a promising approach for autonomous driving, which learns a latent dynamics model and uses it to train a planning policy.

• To speed up the learning process, the pretrain-finetune paradigm is often used, where online RL is initialized by a pretrained model and a policy learned offline.
• However, naively performing such initialization in RL may result in dramatic performance degradation during the online interactions in the new task.
• To tackle this challenge, we first analyze the performance degradation and identify two primary root causes therein: the mismatch of the planning policy and the mismatch of the dynamics model, due to distribution shift.