RefAV: Towards Planning-Centric Scenario Mining

Understand

Autonomous Vehicles (AVs) collect and pseudo-label terabytes of multi-modal data localized to HD maps during normal fleet testing.

• However, identifying interesting and safety-critical scenarios from uncurated driving logs remains a significant challenge.
• Traditional scenario mining techniques are error-prone and prohibitively time-consuming, often relying on hand-crafted structured queries.
• In this work, we revisit spatio-temporal scenario mining through the lens of recent vision-language models (VLMs) to detect whether a described scenario occurs in a driving log and, if so, precisely localize it in both time and space.