Reducing Language Biases in Visual Question Answering with Visually-Grounded Question Encoder

Understand

Recent studies have shown that current VQA models are heavily biased on the language priors in the train set to answer the question, irrespective of the image.

• E.g., overwhelmingly answer "what sport is" as "tennis" or "what color banana" as "yellow." This behavior restricts them from real-world application scenarios.
• In this work, we propose a novel model-agnostic question encoder, Visually-Grounded Question Encoder (VGQE), for VQA that reduces this effect.
• VGQE utilizes both visual and language modalities equally while encoding the question.