Closing the Loop: Motion Prediction Models beyond Open-Loop Benchmarks

Understand

Fueled by motion prediction competitions and benchmarks, recent years have seen the emergence of increasingly large learning based prediction models, many with millions of parameters, focused on improving open-loop prediction accuracy by mere centimeters.

• However, these benchmarks fail to assess whether such improvements translate to better performance when integrated into an autonomous driving stack.
• In this work, we systematically evaluate the interplay between state-of-the-art motion predictors and motion planners.
• Our results show that higher open-loop accuracy does not always correlate with better closed-loop driving behavior and that other factors, such as temporal consistency of predictions and planner compatibility, also play a critical role.