Robust Online Video Instance Segmentation with Track Queries

Understand

Recently, transformer-based methods have achieved impressive results on Video Instance Segmentation (VIS).

• However, most of these top-performing methods run in an offline manner by processing the entire video clip at once to predict instance mask volumes.
• This makes them incapable of handling the long videos that appear in challenging new video instance segmentation datasets like UVO and OVIS.
• We propose a fully online transformer-based video instance segmentation model that performs comparably to top offline methods on the YouTube-VIS 2019 benchmark and considerably outperforms them on UVO and OVIS.