CFSum: A Coarse-to-Fine Contribution Network for Multimodal Summarization

Understand

Multimodal summarization usually suffers from the problem that the contribution of the visual modality is unclear.

• Existing multimodal summarization approaches focus on designing the fusion methods of different modalities, while ignoring the adaptive conditions under which visual modalities are useful.
• Therefore, we propose a novel Coarse-to-Fine contribution network for multimodal Summarization (CFSum) to consider different contributions of images for summarization.
• First, to eliminate the interference of useless images, we propose a pre-filter module to abandon useless images.