Transfer Learning for Low-Resource Neural Machine Translation

Understand

The encoder-decoder framework for neural machine translation (NMT) has been shown effective in large data scenarios, but is much less effective for low-resource languages.

• We present a transfer learning method that significantly improves Bleu scores across a range of low-resource languages.
• Our key idea is to first train a high-resource language pair (the parent model), then transfer some of the learned parameters to the low-resource pair (the child model) to initialize and constrain training.
• Using our transfer learning method we improve baseline NMT models by an average of 5.6 Bleu on four low-resource language pairs.