CLIP also Understands Text: Prompting CLIP for Phrase Understanding

Understand

Contrastive Language-Image Pretraining (CLIP) efficiently learns visual concepts by pre-training with natural language supervision.

• CLIP and its visual encoder have been explored on various vision and language tasks and achieve strong zero-shot or transfer learning performance.
• However, the application of its text encoder solely for text understanding has been less explored.
• In this paper, we find that the text encoder of CLIP actually demonstrates strong ability for phrase understanding, and can even significantly outperform popular language models such as BERT with a properly designed prompt.