High-Order Attention Models for Visual Question Answering

Understand

The quest for algorithms that enable cognitive abilities is an important part of machine learning.

• A common trait in many recently investigated cognitive-like tasks is that they take into account different data modalities, such as visual and textual input.
• In this paper we propose a novel and generally applicable form of attention mechanism that learns high-order correlations between various data modalities.
• We show that high-order correlations effectively direct the appropriate attention to the relevant elements in the different data modalities that are required to solve the joint task.