Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation

Understand

The popularity of image sharing on social media and the engagement it creates between users reflects the important role that visual context plays in everyday conversations.

• We present a novel task, Image-Grounded Conversations (IGC), in which natural-sounding conversations are generated about a shared image.
• To benchmark progress, we introduce a new multiple-reference dataset of crowd-sourced, event-centric conversations on images.
• IGC falls on the continuum between chit-chat and goal-directed conversation models, where visual grounding constrains the topic of conversation to event-driven utterances.