Multimodal Residual Learning for Visual QA

Understand

Deep neural networks continue to advance the state-of-the-art of image recognition tasks with various methods.

• However, applications of these methods to multimodality remain limited.
• We present Multimodal Residual Networks (MRN) for the multimodal residual learning of visual question-answering, which extends the idea of the deep residual learning.
• Unlike the deep residual learning, MRN effectively learns the joint representation from vision and language information.