Combining the Best of Graphical Models and ConvNets for Semantic Segmentation

Understand

We present a two-module approach to semantic segmentation that incorporates Convolutional Networks (CNNs) and Graphical Models.

• Graphical models are used to generate a small (5-30) set of diverse segmentations proposals, such that this set has high recall.
• Since the number of required proposals is so low, we can extract fairly complex features to rank them.
• Our complex feature of choice is a novel CNN called SegNet, which directly outputs a (coarse) semantic segmentation.