Annotation Free Semantic Segmentation with Vision Foundation Models

Understand

Semantic Segmentation is one of the most challenging vision tasks, usually requiring large amounts of training data with expensive pixel level annotations.

• With the success of foundation models and especially vision-language models, recent works attempt to achieve zeroshot semantic segmentation while requiring either large-scale training or additional image/pixel level annotations.
• In this work, we generate free annotations for any semantic segmentation dataset using existing foundation models.
• We use CLIP to detect objects and SAM to generate high quality object masks.