Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input

Understand

Non-autoregressive translation (NAT) models, which remove the dependence on previous target tokens from the inputs of the decoder, achieve significantly inference speedup but at the cost of inferior accuracy compared to autoregressive translation (AT) models.

• Previous work shows that the quality of the inputs of the decoder is important and largely impacts the model accuracy.
• In this paper, we propose two methods to enhance the decoder inputs so as to improve NAT models.
• The first one directly leverages a phrase table generated by conventional SMT approaches to translate source tokens to target tokens, which are then fed into the decoder as inputs.