FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension

Understand

This paper introduces a new neural structure called FusionNet, which extends existing attention approaches from three perspectives.

• First, it puts forward a novel concept of "history of word" to characterize attention information from the lowest word-level embedding up to the highest semantic-level representation.
• Second, it introduces an improved attention scoring function that better utilizes the "history of word" concept.
• Third, it proposes a fully-aware multi-level attention mechanism to capture the complete information in one text (such as a question) and exploit it in its counterpart (such as context or passage) layer by layer.