Audio-Visual Segmentation via Unlabeled Frame Exploitation

Understand

Audio-visual segmentation (AVS) aims to segment the sounding objects in video frames.

• Although great progress has been witnessed, we experimentally reveal that current methods reach marginal performance gain within the use of the unlabeled frames, leading to the underutilization issue.
• To fully explore the potential of the unlabeled frames for AVS, we explicitly divide them into two categories based on their temporal characteristics, i.e., neighboring frame (NF) and distant frame (DF).
• NFs, temporally adjacent to the labeled frame, often contain rich motion information that assists in the accurate localization of sounding objects.