2015

Fully Connected Deep Structured Networks

Schwing, Alexander G., Urtasun, Raquel

Understand

Convolutional neural networks with many layers have recently been shown to achieve excellent results on many high-level tasks such as image classification, object detection and more recently also semantic segmentation.

Particularly for semantic segmentation, a two-stage procedure is often employed.
Hereby, convolutional networks are trained to provide good local pixel-wise features for the second step being traditionally a more global graphical model.
In this work we unify this two-stage process into a single joint training algorithm.

Built on

Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
J. Pearl · 1988
Earlier work this paper cites.
Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters
J. S. Bridle · 1990
Earlier work this paper cites.
Global training of document processing systems using graph transformer networks
L. Bottou, Y. Bengio, and Y. LeCun · 1997
Earlier work this paper cites.
A Wavelet Tour of Signal Processing
S. Mallat · 1999
Earlier work this paper cites.
Tree-based reparameterization framework for analysis of sum-product and related algorithms
M. J. Wainwright, T. Jaakkola, and A. S. Willsky · 2003
Earlier work this paper cites.
The Concave-Convex Procedure (CCCP)
A. L. Yuille and A. Rangarajan · 2003
Earlier work this paper cites.
MAP Estimation, Linear Programming and Belief Propagation with Convex Free Energies
Y. Weiss, C. Yanover, and T. Meltzer · 2007
Earlier work this paper cites.
Conditional random fields for integrating local discriminative classifiers
J. Morris and E. Fosler-Lussier · 2008
Earlier work this paper cites.
Graphical models, exponential families and variational inference
M. J. Wainwright and M. I. Jordan · 2008
Earlier work this paper cites.
Probabilistic Graphical Models: Principles and Techniques
D. Koller and N. Friedman · 2009
Earlier work this paper cites.
Conditional Neural Fields
J. Peng, L. Bo, and J. Xu · 2009
Earlier work this paper cites.
Neural conditional random fields
T.-M.-T. Do and T. Artieres · 2010
Earlier work this paper cites.
Backpropagation training for multilayer conditional random field based phone recognition
R. Prabhavalkar and E. Fosler-Lussier · 2010
Earlier work this paper cites.
Natural language processing (almost) from scratch
R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa · 2011
Earlier work this paper cites.

Similar

Semantic Contours from Inverse Detectors
B. Hariharan, P. Arbeláez, L. Bourdev, S. Maji, and J. Malik · 2011
Cited alongside, same era.
Efficient inference in fully connected CRFs with Gaussian edge potentials
P. Krähenbühl and V. Koltun · 2011
Cited alongside, same era.
The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman · 2012
Cited alongside, same era.
A conditional neural fields model for protein threading
J. Ma, J. Peng, S. Wang, and J. Xu · 2012
Cited alongside, same era.
Improved initialization and Gaussian mixture pairwise terms for dense random fields with mean-field inference
V. Vineet, J. Warrell, P. Sturgess, and P. H. S. Torr · 2012
Cited alongside, same era.
Filter-based mean-field inference for random fields with higher-order terms and product label-spaces
V. Vineet, J. Warrell, and P. H. S. Torr · 2012
Cited alongside, same era.
Structured Learning via Logistic Regression
J. Domke · 2013
Cited alongside, same era.
Fast image scanning with deep max-pooling convolutional neural networks
A. Guisti, D. Ciresan, J. Masci, L. Gambardella, and J. Schmidhuber · 2013
Cited alongside, same era.
Parameter Learning and Convergent Inference for Dense Random Fields
P. Krähenbühl and V. Koltun · 2013
Cited alongside, same era.
ImageNet Classification with Deep Convolutional Neural Networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2013
Cited alongside, same era.
Learning Deep Structured Models
Original
L.-C. Chen, A. G. Schwing, A. L. Yuille, and R. Urtasun · 2014
Cited alongside, same era.
Long-term recurrent convolutional networks for visual recognition and description
Original
J. Donahue, L. A. Hendrikcs, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell · 2014
Cited alongside, same era.

Then

Deep visual-semantic alignments for generating image descriptions
Original
A. Karpathy and L. Fei-Fei · 2014
Later among the works it cites.
Multi- modal neural language models
R. Kiros, R. Salahutdinov, and R. Zemel · 2014
Later among the works it cites.
High Order Regularization for Semi-Supervised Learning of Structured Output Problems
Y. Li and R. Zemel · 2014
Later among the works it cites.
Fully Convolutional Networks for Semantic Segmentation
Original
J. Long, E. Shelhamer, and T. Darrell · 2014
Later among the works it cites.
Deep captioning with multimodal recurrent neural networks (m-rnn)
Original
J. Mao, W. Xu, Y. Yang, J. Wang, and A. L. Yuille · 2014
Later among the works it cites.
ImageNet Large Scale Visual Recognition Challenge, 2014
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei · 2014
Later among the works it cites.
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun · 2014
Later among the works it cites.
Very Deep Convolutional Networks for Large-Scale Image Recognition
Original
K. Simonyan and A. Zisserman · 2014
Later among the works it cites.
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
J. Tompson, A. Jain, Y. LeCun, and C. Bregler · 2014
Later among the works it cites.
Show and tell: A neural image caption generator
Original
O. Vinyals, A. Toshev, S. Bengio, and D. Erhan · 2014
Later among the works it cites.
Computing the Stereo Matching Cost with a Convolutional Neural Network
Original
J. Žbontar and Y. LeCun · 2014
Later among the works it cites.
Semantic ImageSegmentation with Deep Convolutional Nets and Fully Connected CRFs
Original
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2015
Closest in time.
Conditional Random Fields as Recurrent Neural Networks
Original
S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. S. Torr · 2015
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…