Sense, Imagine, Act: Multimodal Perception Improves Model-Based Reinforcement Learning for Head-to-Head Autonomous Racing

Understand

Model-based reinforcement learning (MBRL) techniques have recently yielded promising results for real-world autonomous racing using high-dimensional observations.

• MBRL agents, such as Dreamer, solve long-horizon tasks by building a world model and planning actions by latent imagination.
• This approach involves explicitly learning a model of the system dynamics and using it to learn the optimal policy for continuous control over multiple timesteps.
• As a result, MBRL agents may converge to sub-optimal policies if the world model is inaccurate.