A Comprehensive Evaluation of Four End-to-End AI Autopilots Using CCTest and the Carla Leaderboard

Understand

End-to-end AI autopilots for autonomous driving systems have emerged as a promising alternative to traditional modular autopilots, offering the potential to reduce development costs and mitigate defects arising from module composition.

• However, they suffer from the well-known problems of AI systems such as non-determinism, non-explainability, and anomalies.
• This naturally raises the question of their evaluation and, in particular, their comparison with existing modular solutions.
• This work extends a study of the critical configuration testing (CCTest) approach that has been applied to four open modular autopilots.