Unsupervised Neural Machine Translation with SMT as Posterior Regularization

Understand

Without real bilingual corpus available, unsupervised Neural Machine Translation (NMT) typically requires pseudo parallel data generated with the back-translation method for the model training.

• However, due to weak supervision, the pseudo data inevitably contain noises and errors that will be accumulated and reinforced in the subsequent training process, leading to bad translation performance.
• To address this issue, we introduce phrase based Statistic Machine Translation (SMT) models which are robust to noisy data, as posterior regularizations to guide the training of unsupervised NMT models in the iterative back-translation process.
• Our method starts from SMT models built with pre-trained language models and word-level translation tables inferred from cross-lingual embeddings.