Studying Image Diffusion Features for Zero-Shot Video Object Segmentation

Understand

This paper investigates the use of large-scale diffusion models for Zero-Shot Video Object Segmentation (ZS-VOS) without fine-tuning on video data or training on any image segmentation data.

• While diffusion models have demonstrated strong visual representations across various tasks, their direct application to ZS-VOS remains underexplored.
• Our goal is to find the optimal feature extraction process for ZS-VOS by identifying the most suitable time step and layer from which to extract features.
• We further analyze the affinity of these features and observe a strong correlation with point correspondences.