FreeSeg: Free Mask from Interpretable Contrastive Language-Image Pretraining for Semantic Segmentation

Understand

Fully supervised semantic segmentation learns from dense masks, which requires heavy annotation cost for closed set.

• In this paper, we use natural language as supervision without any pixel-level annotation for open world segmentation.
• We call the proposed framework as FreeSeg, where the mask is freely available from raw feature map of pretraining model.
• Compared with zero-shot or openset segmentation, FreeSeg doesn't require any annotated masks, and it widely predicts categories beyond class-agnostic unsupervised segmentation.