$C^3$: Compositional Counterfactual Contrastive Learning for Video-grounded Dialogues

Understand

Video-grounded dialogue systems aim to integrate video understanding and dialogue understanding to generate responses that are relevant to both the dialogue and video context.

• Most existing approaches employ deep learning models and have achieved remarkable performance, given the relatively small datasets available.
• However, the results are partly accomplished by exploiting biases in the datasets rather than developing multimodal reasoning, resulting in limited generalization.
• In this paper, we propose a novel approach of Compositional Counterfactual Contrastive Learning ($C^3$) to develop contrastive training between factual and counterfactual samples in video-grounded dialogues.