Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation

Understand

There has been recent success in pre-training on monolingual data and fine-tuning on Machine Translation (MT), but it remains unclear how to best leverage a pre-trained model for a given MT task.

• This paper investigates the benefits and drawbacks of freezing parameters, and adding new ones, when fine-tuning a pre-trained model on MT.
• We focus on 1) Fine-tuning a model trained only on English monolingual data, BART.
• 2) Fine-tuning a model trained on monolingual data from 25 languages, mBART.