Improved baselines for vision-language pre-training

Understand

Contrastive learning has emerged as an efficient framework to learn multimodal representations.

• CLIP, a seminal work in this area, achieved impressive results by training on paired image-text data using the contrastive loss.
• Recent work claims improvements over CLIP using additional non-contrastive losses inspired from self-supervised learning.
• However, it is sometimes hard to disentangle the contribution of these additional losses from other implementation details, e.g., data augmentation or regularization techniques, used to train the model.