2016

RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation

Lin, Guosheng, Milan, Anton, Shen, Chunhua et al.

Understand

Recently, very deep convolutional neural networks (CNNs) have shown outstanding performance in object recognition and have also been the first choice for dense classification problems such as semantic segmentation.

However, repeated subsampling operations like pooling or convolution striding in deep CNNs lead to a significant decrease in the initial image resolution.
Here, we present RefineNet, a generic multi-path refinement network that explicitly exploits all the information available along the down-sampling process to enable high-resolution prediction using long-range residual connections.
In this way, the deeper layers that capture high-level semantic features can be directly refined using fine-grained features from earlier convolutions.

Built on

The pascal visual object classes (voc) challenge
M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman · 2010
Earlier work this paper cites.
Semantic contours from inverse detectors
B. Hariharan, P. Arbelaez, L. D. Bourdev, S. Maji, and J. Malik · 2011
Earlier work this paper cites.
Sift flow: Dense correspondence across scenes and its applications
C. Liu, J. Yuen, and A. Torralba · 2011
Earlier work this paper cites.
Semantic segmentation with second-order pooling
J. Carreira, R. Caseiro, J. Batista, and C. Sminchisescu · 2012
Earlier work this paper cites.
Efficient inference in fully connected CRFs with Gaussian edge potentials
P. Krähenbühl and V. Koltun · 2012
Earlier work this paper cites.
Rgb-(d) scene labeling: Features and algorithms
X. Ren, L. Bo, and D. Fox · 2012
Earlier work this paper cites.
Indoor segmentation and support inference from rgbd images
N. Silberman, D. Hoiem, P. Kohli, and R. Fergus · 2012
Earlier work this paper cites.
Restoring an image taken through a window covered with dirt or rain
D. Eigen, D. Krishnan, and R. Fergus · 2013
Earlier work this paper cites.
Perceptual organization and recognition of indoor scenes from rgb-d images
S. Gupta, P. Arbelaez, and J. Malik · 2013
Earlier work this paper cites.
Detect what you can: Detecting and representing objects using holistic models and body parts
X. Chen, R. Mottaghi, X. Liu, S. Fidler, R. Urtasun, and A. Yuille · 2014
Earlier work this paper cites.
Learning a deep convolutional network for image super-resolution
C. Dong, C. C. Loy, K. He, and X. Tang · 2014
Earlier work this paper cites.
Depth map prediction from a single image using a multi-scale deep network
D. Eigen, C. Puhrsch, and R. Fergus · 2014
Earlier work this paper cites.
Rich feature hierarchies for accurate object detection and semantic segmentation
R. B. Girshick, J. Donahue, T. Darrell, and J. Malik · 2014
Earlier work this paper cites.
Learning rich features from RGB-D images for object detection and segmentation
S. Gupta, R. Girshick, P. Arbeláez, and J. Malik · 2014
Earlier work this paper cites.
Hypercolumns for object segmentation and fine-grained localization
B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik · 2014
Earlier work this paper cites.
Simultaneous detection and segmentation
B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik · 2014
Earlier work this paper cites.

Similar

Microsoft COCO: Common objects in context
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick · 2014
Cited alongside, same era.
The role of context for object detection and semantic segmentation in the wild
R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, et al · 2014
Cited alongside, same era.
MatConvNet – convolutional neural networks for matlab, 2014
A. Vedaldi and K. Lenc · 2014
Cited alongside, same era.
Segnet: A deep convolutional encoder-decoder architecture for image segmentation
V. Badrinarayanan, A. Kendall, and R. Cipolla · 2015
Cited alongside, same era.
Semantic image segmentation with deep convolutional nets and fully connected CRFs
L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2015
Cited alongside, same era.
Attention to scale: Scale-aware semantic image segmentation
Original
L.-C. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille · 2015
Cited alongside, same era.
BoxSup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation
J. Dai, K. He, and J. Sun · 2015
Cited alongside, same era.
Convolutional feature masking for joint object and stuff segmentation
J. Dai, K. He, and J. Sun · 2015
Cited alongside, same era.
Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture
D. Eigen and R. Fergus · 2015
Cited alongside, same era.
Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding
Original
A. Kendall, V. Badrinarayanan, and R. Cipolla · 2015
Cited alongside, same era.
Semantic object parsing with local-global long short-term memory
Original
X. Liang, X. Shen, D. Xiang, J. Feng, L. Lin, and S. Yan · 2015
Cited alongside, same era.
Deep convolutional neural fields for depth estimation from a single image
F. Liu, C. Shen, and G. Lin · 2015
Cited alongside, same era.

Then

U-net: Convolutional networks for biomedical image segmentation
O. Ronneberger, P. Fischer, and T. Brox · 2015
Later among the works it cites.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2015
Later among the works it cites.
Sun rgb-d: A rgb-d scene understanding benchmark suite
S. Song, S. P. Lichtenberg, and J. Xiao · 2015
Later among the works it cites.
Zoom better to see clearer: Human and object parsing with hierarchical auto-zoom net
Original
F. Xia, P. Wang, L.-C. Chen, and A. L. Yuille · 2015
Later among the works it cites.
Multi-scale context aggregation by dilated convolutions
F. Yu and V. Koltun · 2015
Later among the works it cites.
Conditional random fields as recurrent neural networks
S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. Torr · 2015
Later among the works it cites.
Higher order conditional random fields in deep neural networks
A. Arnab, S. Jayasumana, S. Zheng, and P. H. Torr · 2016
Closest in time.
Fast, exact and multi-scale inference for semantic image segmentation with deep gaussian crfs
S. Chandra and I. Kokkinos · 2016
Closest in time.
DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs
Original
L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2016
Closest in time.
The cityscapes dataset for semantic urban scene understanding
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele · 2016
Closest in time.
Laplacian pyramid reconstruction and refinement for semantic segmentation
G. Ghiasi and C. C. Fowlkes · 2016
Closest in time.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Closest in time.
Identity mappings in deep residual networks
Original
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Closest in time.
Semantic object parsing with graph lstm
Original
X. Liang, X. Shen, J. Feng, L. Lin, and S. Yan · 2016
Closest in time.
Efficient piecewise training of deep structured models for semantic segmentation
G. Lin, C. Shen, A. van den Hengel, and I. Reid · 2016
Closest in time.
Semantic understanding of scenes through the ADE20K dataset
Original
B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba · 2016
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…