Supervised Attentions for Neural Machine Translation

Understand

In this paper, we improve the attention or alignment accuracy of neural machine translation by utilizing the alignments of training sentence pairs.

• We simply compute the distance between the machine attentions and the "true" alignments, and minimize this cost in the training procedure.
• Our experiments on large-scale Chinese-to-English task show that our model improves both translation and alignment qualities significantly over the large-vocabulary neural machine translation system, and even beats a state-of-the-art traditional syntax-based system.