LiT: Zero-Shot Transfer with Locked-image text Tuning

Understand

This paper presents contrastive-tuning, a simple method employing contrastive training to align image and text models while still taking advantage of their pre-training.

• In our empirical study we find that locked pre-trained image models with unlocked text models work best.
• We call this instance of contrastive-tuning "Locked-image Tuning" (LiT), which just teaches a text model to read out good representations from a pre-trained image model for new tasks.
• A LiT model gains the capability of zero-shot transfer to new vision tasks, such as image classification or retrieval.