Explicit Cross-lingual Pre-training for Unsupervised Machine Translation

Understand

Pre-training has proven to be effective in unsupervised machine translation due to its ability to model deep context information in cross-lingual scenarios.

• However, the cross-lingual information obtained from shared BPE spaces is inexplicit and limited.
• In this paper, we propose a novel cross-lingual pre-training method for unsupervised machine translation by incorporating explicit cross-lingual training signals.
• Specifically, we first calculate cross-lingual n-gram embeddings and infer an n-gram translation table from them.