MVP: Multi-task Supervised Pre-training for Natural Language Generation

Understand

Pre-trained language models (PLMs) have achieved remarkable success in natural language generation (NLG) tasks.

• Up to now, most NLG-oriented PLMs are pre-trained in an unsupervised manner using the large-scale general corpus.
• In the meanwhile, an increasing number of models pre-trained with labeled data (i.e.
• "supervised pre-training") showcase superior performance compared to unsupervised pre-trained models.