Leveraging Vision-Language Models for Visual Grounding and Analysis of Automotive UI

Understand

Modern automotive infotainment systems necessitate intelligent and adaptive solutions to manage frequent User Interface (UI) updates and diverse design variations.

• This work introduces a vision-language framework to facilitate the understanding of and interaction with automotive UIs, enabling seamless adaptation across different UI designs.
• To support research in this field, AutomotiveUI-Bench-4K, an open-source dataset comprising 998 images with 4,208 annotations, is also released.
• Additionally, a data pipeline for generating training data is presented.