Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

Understand

The tremendous success of CLIP (Radford et al., 2021) has promoted the research and application of contrastive learning for vision-language pretraining.

• In this work, we construct a large-scale dataset of image-text pairs in Chinese, where most data are retrieved from publicly available datasets, and we pretrain Chinese CLIP models on the new dataset.
• We develop 5 Chinese CLIP models of multiple sizes, spanning from 77 to 958 million parameters.
• Furthermore, we propose a two-stage pretraining method, where the model is first trained with the image encoder frozen and then trained with all parameters being optimized, to achieve enhanced model performance.