2014

Learning Deep Structured Models

Chen, Liang-Chieh, Schwing, Alexander G., Yuille, Alan L. et al.

Understand

Many problems in real-world applications involve predicting several random variables which are statistically related.

Markov random fields (MRFs) are a great mathematical tool to encode such relationships.
The goal of this paper is to combine MRFs with deep learning algorithms to estimate complex representations while taking into account the dependencies between the output random variables.
Towards this goal, we propose a training algorithm that is able to learn structured models jointly with deep features that form the MRF potentials.

Built on

Boltzmann Machines: Constraint Satisfaction Networks that Learn
Hinton, G. E., Sejnowski, T. J., and Ackley, D. H · 1984
Earlier work this paper cites.
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
Pearl, J · 1988
Earlier work this paper cites.
Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters
Bridle, J. S · 1990
Earlier work this paper cites.
Global training of document processing systems using graph transformer networks
Bottou, L., Bengio, Y., and LeCun, Y · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P · 1998
Earlier work this paper cites.
Fractional belief propagation
Wiegerinck, W. and Heskes, T · 2003
Earlier work this paper cites.
A new class of upper bounds on the log partition function
Wainwright, M. J., Jaakkola, T., and Willsky, A. S · 2005
Earlier work this paper cites.
Constructing free-energy approximations and generalized belief propagation algorithms
Yedidia, J. S., Freeman, W. T., and Weiss, Y · 2005
Earlier work this paper cites.
Reducing the dimensionality of data with neural networks
Hinton, G. E. and Salakhutdinov, R. R · 2006
Earlier work this paper cites.
Greedy Layer-Wise Training of Deep Networks
Bengio, Y., Lamblin, P., Popovici, D., and Larochelle, H · 2007
Earlier work this paper cites.
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., and Bengio, Y · 2007
Earlier work this paper cites.
MAP Estimation, Linear Programming and Belief Propagation with Convex Free Energies
Weiss, Y., Yanover, C., and Meltzer, T · 2007
Earlier work this paper cites.
Conditional random fields for integrating local discriminative classifiers
Morris, J. and Fosler-Lussier, E · 2008
Earlier work this paper cites.
Graphical Models, Exponential Families and Variational Inference
Wainwright, M. J. and Jordan, M. I · 2008
Earlier work this paper cites.

Similar

Character recognition in natural images
de Campos, T. E., Babu, B. R., and Varma, M · 2009
Cited alongside, same era.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Cited alongside, same era.
Probabilistic Graphical Models: Principles and Techniques
Koller, D. and Friedman, N · 2009
Cited alongside, same era.
Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations
Lee, H., Grosse, R., Ranganath, R., and Ng, A. Y · 2009
Cited alongside, same era.
Convergent Message Passing Algorithms: a unifying view
Meltzer, T., Globerson, A., and Weiss, Y · 2009
Cited alongside, same era.
Conditional Neural Fields
Peng, J., Bo, L., and Xu, J · 2009
Cited alongside, same era.
Neural conditional random fields
Do, T.-M.-T. and Artieres, T · 2010
Cited alongside, same era.
A Primal-Dual Message-Passing Algorithm for Approximated Large Scale Structured Prediction
Hazan, T. and Urtasun, R · 2010
Cited alongside, same era.
Learning Efficiently with Approximate inference via Dual Losses
Meshi, O., Sontag, D., Jaakkola, T., and Globerson, A · 2010
Cited alongside, same era.
Backpropagation training for multilayer conditional random field based phone recognition
Prabhavalkar, R. and Fosler-Lussier, E · 2010
Cited alongside, same era.
Natural language processing (almost) from scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., and Kuksa, P · 2011
Cited alongside, same era.
Decision tree fields
Nowozin, S., Rother, C., Bagon, S., Sharp, T., Yao, B., and Kohli, P · 2011
Cited alongside, same era.

Then

Convolutional-Recursive Deep Learning for 3D Object Classification
Socher, R., Huval, B., Bhat, B., Manning, C. D., and Ng, A. Y · 2012
Later among the works it cites.
Structured Learning via Logistic Regression
Domke, J · 2013
Later among the works it cites.
Caffe: An Open Source Convolutional Architecture for Fast Feature Embedding
Jia, Y · 2013
Later among the works it cites.
ImageNet Classification with Deep Convolutional Neural Networks
Krizhevsky, A., Sutskever, I., and Hinton, G. E · 2013
Later among the works it cites.
Inference and Learning Algorithms with Applications to 3D Indoor Scene Understanding
Schwing, A. G · 2013
Later among the works it cites.
Large-Scale Object Classification using Label Relation Graphs
Deng, J., Ding, N., Jia, Y., Frome, A., Murphy, K., Bengio, S., Li, Y., Neven, H., and Adam, H · 2014
Closest in time.
Understanding Deep Architectures using a Recursive Convolutional Network
Eigen, D., Rolfe, J., Fergus, R., and LeCun, Y · 2014
Closest in time.
Rich feature hierarchies for accurate object detection and semantic segmentation
Girshick, R., Donahue, J., Darrell, T., and Malik, J · 2014
Closest in time.
Simultaneous detection and segmentation
Hariharan, B., Arbeláez, P., Girshick, R., and Malik, J · 2014
Closest in time.
High Order Regularization for Semi-Supervised Learning of Structured Output Problems
Li, Y. and Zemel, R · 2014
Closest in time.
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
Tompson, J., Jain, A., LeCun, Y., and Bregler, C · 2014
Closest in time.
Tell me what you see and I will show you where it is
Xu, J., Schwing, A. G., and Urtasun, R · 2014
Closest in time.
Visualizing and Understanding Convolutional Networks
Zeiler, M. D. and Fergus, R · 2014
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…