The Missing Ingredient in Zero-Shot Neural Machine Translation

Understand

Multilingual Neural Machine Translation (NMT) models are capable of translating between multiple source and target languages.

• Despite various approaches to train such models, they have difficulty with zero-shot translation: translating between language pairs that were not together seen during training.
• In this paper we first diagnose why state-of-the-art multilingual NMT models that rely purely on parameter sharing, fail to generalize to unseen language pairs.
• We then propose auxiliary losses on the NMT encoder that impose representational invariance across languages.