Optimizing Transformer for Low-Resource Neural Machine Translation

Understand

Language pairs with limited amounts of parallel data, also known as low-resource languages, remain a challenge for neural machine translation.

• While the Transformer model has achieved significant improvements for many language pairs and has become the de facto mainstream architecture, its capability under low-resource conditions has not been fully investigated yet.
• Our experiments on different subsets of the IWSLT14 training data show that the effectiveness of Transformer under low-resource conditions is highly dependent on the hyper-parameter settings.
• Our experiments show that using an optimized Transformer for low-resource conditions improves the translation quality up to 7.3 BLEU points compared to using the Transformer default settings.