Analyzing Uncertainty in Neural Machine Translation

Understand

Machine translation is a popular test bed for research in neural sequence-to-sequence models but despite much recent research, there is still a lack of understanding of these models.

• Practitioners report performance degradation with large beams, the under-estimation of rare words and a lack of diversity in the final translations.
• Our study relates some of these issues to the inherent uncertainty of the task, due to the existence of multiple valid translations for a single source sentence, and to the extrinsic uncertainty caused by noisy training data.
• We propose tools and metrics to assess how uncertainty in the data is captured by the model distribution and how it affects search strategies that generate translations.