Graph-Structured Representations for Visual Question Answering

Understand

This paper proposes to improve visual question answering (VQA) with structured representations of both scene contents and questions.

• A key challenge in VQA is to require joint reasoning over the visual and text domains.
• The predominant CNN/LSTM-based approach to VQA is limited by monolithic vector representations that largely ignore structure in the scene and in the form of the question.
• CNN feature vectors cannot effectively capture situations as simple as multiple object instances, and LSTMs process questions as series of words, which does not reflect the true complexity of language structure.