Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering

Understand

Learning effective fusion of multi-modality features is at the heart of visual question answering.

• We propose a novel method of dynamically fusing multi-modal features with intra- and inter-modality information flow, which alternatively pass dynamic information between and across the visual and language modalities.
• It can robustly capture the high-level interactions between language and vision domains, thus significantly improves the performance of visual question answering.
• We also show that the proposed dynamic intra-modality attention flow conditioned on the other modality can dynamically modulate the intra-modality attention of the target modality, which is vital for multimodality feature fusion.