Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling

Understand

Contrastive Vision-Language Pre-training, known as CLIP, has provided a new paradigm for learning visual representations by using large-scale contrastive image-text pairs.

• It shows impressive performance on zero-shot knowledge transfer to downstream tasks.
• To further enhance CLIP's few-shot capability, CLIP-Adapter proposed to fine-tune a lightweight residual feature adapter and significantly improves the performance for few-shot classification.
• However, such a process still needs extra training and computational resources.