Zoom-shot: Fast and Efficient Unsupervised Zero-Shot Transfer of CLIP to Vision Encoders with Multimodal Loss

Understand

The fusion of vision and language has brought about a transformative shift in computer vision through the emergence of Vision-Language Models (VLMs).

• However, the resource-intensive nature of existing VLMs poses a significant challenge.
• We need an accessible method for developing the next generation of VLMs.
• To address this issue, we propose Zoom-shot, a novel method for transferring the zero-shot capabilities of CLIP to any pre-trained vision encoder.