Linguistic Query-Guided Mask Generation for Referring Image Segmentation

Understand

Referring image segmentation aims to segment the image region of interest according to the given language expression, which is a typical multi-modal task.

• Existing methods either adopt the pixel classification-based or the learnable query-based framework for mask generation, both of which are insufficient to deal with various text-image pairs with a fix number of parametric prototypes.
• In this work, we propose an end-to-end framework built on transformer to perform Linguistic query-Guided mask generation, dubbed LGFormer.
• It views the linguistic features as query to generate a specialized prototype for arbitrary input image-text pair, thus generating more consistent segmentation results.