Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation

Understand

Non-Autoregressive Neural Machine Translation (NAT) achieves significant decoding speedup through generating target words independently and simultaneously.

• However, in the context of non-autoregressive translation, the word-level cross-entropy loss cannot model the target-side sequential dependency properly, leading to its weak correlation with the translation quality.
• As a result, NAT tends to generate influent translations with over-translation and under-translation errors.
• In this paper, we propose to train NAT to minimize the Bag-of-Ngrams (BoN) difference between the model output and the reference sentence.