Attention to Scale: Scale-aware Semantic Image Segmentation

Understand

Incorporating multi-scale features in fully convolutional neural networks (FCNs) has been a key element to achieving state-of-the-art performance on semantic image segmentation.

• One common way to extract multi-scale features is to feed multiple resized input images to a shared deep network and then merge the resulting features for pixelwise classification.
• In this work, we propose an attention mechanism that learns to softly weight the multi-scale features at each pixel location.
• We adapt a state-of-the-art semantic image segmentation model, which we jointly train with multi-scale input images and the attention model.