KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature Adaptation of Vision-Language Models

Understand

Image ad understanding is a crucial task with wide real-world applications.

• Although highly challenging with the involvement of diverse atypical scenes, real-world entities, and reasoning over scene-texts, how to interpret image ads is relatively under-explored, especially in the era of foundational vision-language models (VLMs) featuring impressive generalizability and adaptability.
• In this paper, we perform the first empirical study of image ad understanding through the lens of pre-trained VLMs.
• We benchmark and reveal practical challenges in adapting these VLMs to image ad understanding.