Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation

Understand

Transfer learning between different language pairs has shown its effectiveness for Neural Machine Translation (NMT) in low-resource scenario.

• However, existing transfer methods involving a common target language are far from success in the extreme scenario of zero-shot translation, due to the language space mismatch problem between transferor (the parent model) and transferee (the child model) on the source side.
• To address this challenge, we propose an effective transfer learning approach based on cross-lingual pre-training.
• Our key idea is to make all source languages share the same feature space and thus enable a smooth transition for zero-shot translation.