DeVIS: Making Deformable Transformers Work for Video Instance Segmentation

Understand

Video Instance Segmentation (VIS) jointly tackles multi-object detection, tracking, and segmentation in video sequences.

• In the past, VIS methods mirrored the fragmentation of these subtasks in their architectural design, hence missing out on a joint solution.
• Transformers recently allowed to cast the entire VIS task as a single set-prediction problem.
• Nevertheless, the quadratic complexity of existing Transformer-based methods requires long training times, high memory requirements, and processing of low-single-scale feature maps.