DetailCLIP: Injecting Image Details into CLIP's Feature Space

Understand

Although CLIP-like Visual Language Models provide a functional joint feature space for image and text, due to the limitation of the CILP-like model's image input size (e.g., 224), subtle details are lost in the feature representation if we input high-resolution images (e.g., 2240).

• Our proposed framework addresses this issue by generating a single feature representation for a high-resolution image that retains image details from different scales while sharing the same semantic space as the original CLIP.
• An application scenario is remote sensing text-image retrieval, where targets (e.g., vehicles and ships) often appear at tiny scales.
• To achieve this, we develop a feature fusion model that relies on CLIP features extracted from a carefully designed image patch method, dubbed Complete Cover.