WuDaoMM: A large-scale Multi-Modal Dataset for Pre-training models

Understand

Compared with the domain-specific model, the vision-language pre-training models (VLPMs) have shown superior performance on downstream tasks with fast fine-tuning process.

• For example, ERNIE-ViL, Oscar and UNIMO trained VLPMs with a uniform transformers stack architecture and large amounts of image-text paired data, achieving remarkable results on downstream tasks such as image-text reference(IR and TR), vision question answering (VQA) and image captioning (IC) etc.
• During the training phase, VLPMs are always fed with a combination of multiple public datasets to meet the demand of large-scare training data.
• However, due to the unevenness of data distribution including size, task type and quality, using the mixture of multiple datasets for model training can be problematic.