Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering

Understand

In this paper, we present the mQA model, which is able to answer questions about the content of an image.

• The answer can be a sentence, a phrase or a single word.
• Our model contains four components: a Long Short-Term Memory (LSTM) to extract the question representation, a Convolutional Neural Network (CNN) to extract the visual representation, an LSTM for storing the linguistic context in an answer, and a fusing component to combine the information from the first three components and generate the answer.
• We construct a Freestyle Multilingual Image Question Answering (FM-IQA) dataset to train and evaluate our mQA model.