Learning Category- and Instance-Aware Pixel Embedding for Fast Panoptic Segmentation

Understand

Panoptic segmentation (PS) is a complex scene understanding task that requires providing high-quality segmentation for both thing objects and stuff regions.

• Previous methods handle these two classes with semantic and instance segmentation modules separately, following with heuristic fusion or additional modules to resolve the conflicts between the two outputs.
• This work simplifies this pipeline of PS by consistently modeling the two classes with a novel PS framework, which extends a detection model with an extra module to predict category- and instance-aware pixel embedding (CIAE).
• CIAE is a novel pixel-wise embedding feature that encodes both semantic-classification and instance-distinction information.