Temporal Attention Model for Neural Machine Translation

Understand

Attention-based Neural Machine Translation (NMT) models suffer from attention deficiency issues as has been observed in recent research.

• We propose a novel mechanism to address some of these limitations and improve the NMT attention.
• Specifically, our approach memorizes the alignments temporally (within each sentence) and modulates the attention with the accumulated temporal memory, as the decoder generates the candidate translation.
• We compare our approach against the baseline NMT model and two other related approaches that address this issue either explicitly or implicitly.