Cross-Lingual Natural Language Generation via Pre-Training

Understand

In this work we focus on transferring supervision signals of natural language generation (NLG) tasks between multiple languages.

• We propose to pretrain the encoder and the decoder of a sequence-to-sequence model under both monolingual and cross-lingual settings.
• The pre-training objective encourages the model to represent different languages in the shared space, so that we can conduct zero-shot cross-lingual transfer.
• After the pre-training procedure, we use monolingual data to fine-tune the pre-trained model on downstream NLG tasks.