Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering

Understand

Most Outside-Knowledge Visual Question Answering (OK-VQA) systems employ a two-stage framework that first retrieves external knowledge given the visual question and then predicts the answer based on the retrieved content.

• However, the retrieved knowledge is often inadequate.
• Retrievals are frequently too general and fail to cover specific knowledge needed to answer the question.
• Also, the naturally available supervision (whether the passage contains the correct answer) is weak and does not guarantee question relevancy.