Turning a CLIP Model into a Scene Text Detector

Understand

The recent large-scale Contrastive Language-Image Pretraining (CLIP) model has shown great potential in various downstream tasks via leveraging the pretrained vision and language knowledge.

• Scene text, which contains rich textual and visual information, has an inherent connection with a model like CLIP.
• Recently, pretraining approaches based on vision language models have made effective progresses in the field of text detection.
• In contrast to these works, this paper proposes a new method, termed TCM, focusing on Turning the CLIP Model directly for text detection without pretraining process.