2015

Efficient piecewise training of deep structured models for semantic segmentation

Lin, Guosheng, Shen, Chunhua, Hengel, Anton van dan et al.

Understand

Recent advances in semantic image segmentation have mostly been achieved by training deep convolutional neural networks (CNNs).

We show how to improve semantic segmentation through the use of contextual information; specifically, we explore `patch-patch' context between image regions, and `patch-background' context.
For learning from the patch-patch context, we formulate Conditional Random Fields (CRFs) with CNN-based pairwise potential functions to capture semantic correlations between neighboring patches.
Efficient piecewise training of the proposed deep structured model is then applied to avoid repeated expensive CRF inference for back propagation.

Built on

Efficiency of pseudolikelihood estimation for simple Gaussian fields
J. Besag · 1977
Earlier work this paper cites.
Piecewise training for undirected models
C. A. Sutton and A. McCallum · 2005
Earlier work this paper cites.
Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories
S. Lazebnik, C. Schmid, and J. Ponce · 2006
Earlier work this paper cites.
The layout consistent random field for recognizing and segmenting partially occluded objects
J. Winn and J. Shotton · 2006
Earlier work this paper cites.
Objects in context
A. Rabinovich, A. Vedaldi, C. Galleguillos, E. Wiewiora, and S. Belongie · 2007
Earlier work this paper cites.
Learning spatial context: Using stuff to find things
G. Heitz and D. Koller · 2008
Earlier work this paper cites.
The pascal visual object classes (voc) challenge
M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman · 2010
Earlier work this paper cites.
Markov random fields with asymmetric interactions for modelling spatial context in structured scene labelling
D. Heesch and M. Petrou · 2010
Earlier work this paper cites.
Semantic contours from inverse detectors
B. Hariharan, P. Arbelaez, L. D. Bourdev, S. Maji, and J. Malik · 2011
Earlier work this paper cites.
Sift flow: Dense correspondence across scenes and its applications
C. Liu, J. Yuen, and A. Torralba · 2011
Earlier work this paper cites.
Structured learning and prediction in computer vision
S. Nowozin and C. Lampert · 2011
Earlier work this paper cites.
Decision tree fields
S. Nowozin, C. Rother, S. Bagon, T. Sharp, B. Yao, and P. Kohli · 2011
Earlier work this paper cites.
Semantic segmentation with second-order pooling
J. Carreira, R. Caseiro, J. Batista, and C. Sminchisescu · 2012
Earlier work this paper cites.
Regression tree fields—an efficient, non-parametric approach to image labeling problems
J. Jancsary, S. Nowozin, T. Sharp, and C. Rother · 2012
Earlier work this paper cites.
Efficient inference in fully connected CRFs with Gaussian edge potentials
P. Krähenbühl and V. Koltun · 2012
Earlier work this paper cites.
Indoor segmentation and support inference from rgbd images
N. Silberman, D. Hoiem, P. Kohli, and R. Fergus · 2012
Earlier work this paper cites.

Similar

Restoring an image taken through a window covered with dirt or rain
D. Eigen, D. Krishnan, and R. Fergus · 2013
Cited alongside, same era.
Learning hierarchical features for scene labeling
C. Farabet, C. Couprie, L. Najman, and Y. LeCun · 2013
Cited alongside, same era.
Perceptual organization and recognition of indoor scenes from rgb-d images
S. Gupta, P. Arbelaez, and J. Malik · 2013
Cited alongside, same era.
Parameter learning and convergent inference for dense random fields
P. Krähenbühl and V. Koltun · 2013
Cited alongside, same era.
Finding things: Image parsing with regions and per-exemplar detectors
J. Tighe and S. Lazebnik · 2013
Cited alongside, same era.
Context as supervisory signal: Discovering objects with predictable context
C. Doersch, A. Gupta, and A. A. Efros · 2014
Cited alongside, same era.
Learning a deep convolutional network for image super-resolution
C. Dong, C. C. Loy, K. He, and X. Tang · 2014
Cited alongside, same era.
Depth map prediction from a single image using a multi-scale deep network
D. Eigen, C. Puhrsch, and R. Fergus · 2014
Cited alongside, same era.
Rich feature hierarchies for accurate object detection and semantic segmentation
R. B. Girshick, J. Donahue, T. Darrell, and J. Malik · 2014
Cited alongside, same era.
Learning rich features from rgb-d images for object detection and segmentation
S. Gupta, R. Girshick, P. Arbeláez, and J. Malik · 2014
Cited alongside, same era.
Hypercolumns for object segmentation and fine-grained localization
B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik · 2014
Cited alongside, same era.
Simultaneous detection and segmentation
B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik · 2014
Cited alongside, same era.

Then

Joint training of a convolutional network and a graphical model for human pose estimation
J. Tompson, A. Jain, Y. LeCun, and C. Bregler · 2014
Later among the works it cites.
MatConvNet – convolutional neural networks for matlab, 2014
A. Vedaldi and K. Lenc · 2014
Later among the works it cites.
Semantic image segmentation with deep convolutional nets and fully connected CRFs
L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2015
Closest in time.
Learning deep structured models
L.-C. Chen, A. G. Schwing, A. L. Yuille, and R. Urtasun · 2015
Closest in time.
BoxSup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation
J. Dai, K. He, and J. Sun · 2015
Closest in time.
Convolutional feature masking for joint object and stuff segmentation
J. Dai, K. He, and J. Sun · 2015
Closest in time.
Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture
D. Eigen and R. Fergus · 2015
Closest in time.
Deep convolutional neural fields for depth estimation from a single image
F. Liu, C. Shen, and G. Lin · 2015
Closest in time.
Learning depth from single monocular images using deep convolutional neural fields, 2015
Original
F. Liu, C. Shen, G. Lin, and I. Reid · 2015
Closest in time.
Semantic image segmentation via deep parsing network
Z. Liu, X. Li, P. Luo, C. C. Loy, and X. Tang · 2015
Closest in time.
Fully convolutional networks for semantic segmentation
J. Long, E. Shelhamer, and T. Darrell · 2015
Closest in time.
Feedforward semantic segmentation with zoom-out features
M. Mostajabi, P. Yadollahpour, and G. Shakhnarovich · 2015
Closest in time.
Learning deconvolution network for semantic segmentation
H. Noh, S. Hong, and B. Han · 2015
Closest in time.
Fully connected deep structured networks, 2015
Original
A. G. Schwing and R. Urtasun · 2015
Closest in time.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2015
Closest in time.
Conditional random fields as recurrent neural networks
S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. Torr · 2015
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…