Cross-modal Knowledge Reasoning for Knowledge-based Visual Question Answering

Understand

Knowledge-based Visual Question Answering (KVQA) requires external knowledge beyond the visible content to answer questions about an image.

• This ability is challenging but indispensable to achieve general VQA.
• One limitation of existing KVQA solutions is that they jointly embed all kinds of information without fine-grained selection, which introduces unexpected noises for reasoning the correct answer.
• How to capture the question-oriented and information-complementary evidence remains a key challenge to solve the problem.