Phase Conductor on Multi-layered Attentions for Machine Comprehension

Understand

Attention models have been intensively studied to improve NLP tasks such as machine comprehension via both question-aware passage attention model and self-matching attention model.

• Our research proposes phase conductor (PhaseCond) for attention models in two meaningful ways.
• First, PhaseCond, an architecture of multi-layered attention models, consists of multiple phases each implementing a stack of attention layers producing passage representations and a stack of inner or outer fusion layers regulating the information flow.
• Second, we extend and improve the dot-product attention function for PhaseCond by simultaneously encoding multiple question and passage embedding layers from different perspectives.