Single Network Panoptic Segmentation for Street Scene Understanding

Understand

In this work, we propose a single deep neural network for panoptic segmentation, for which the goal is to provide each individual pixel of an input image with a class label, as in semantic segmentation, as well as a unique identifier for specific objects in an image, following instance segmentation.

• Our network makes joint semantic and instance segmentation predictions and combines these to form an output in the panoptic format.
• This has two main benefits: firstly, the entire panoptic prediction is made in one pass, reducing the required computation time and resources; secondly, by learning the tasks jointly, information is shared between the two tasks, thereby improving performance.
• Our network is evaluated on two street scene datasets: Cityscapes and Mapillary Vistas.