Fetching the paper…

Reading the bibliography…

2013

Joint Training Deep Boltzmann Machines for Classification

Goodfellow, Ian J., Courville, Aaron, Bengio, Yoshua

Original
PDF
Post on X

Understand

We introduce a new method for training deep Boltzmann machines jointly.

Prior methods of training DBMs require an initial learning pass that trains the model greedily, one layer at a time, or do not perform well on classification tasks.
In our approach, we train all layers of the DBM simultaneously, using a novel training procedure called multi-prediction training.
The resulting model can either be interpreted as a single generative model trained to maximize a variational approximation to the generalized pseudolikelihood, or as a family of recurrent networks that share parameters and may be approximately averaged together using a novel technique we call the multi-inference trick.

Joint Training Deep Boltzmann Machines for Classification · Around

Built on

On the convergence of Markovian stochastic algorithms with rapidly decreasing ergodicity rates
Younes, L. (1999) · 1999
Earlier work this paper cites.
Annealed importance sampling
Neal, R. M. (2001) · 2001
Earlier work this paper cites.
Representational power of restricted Boltzmann machines and deep belief networks
Le Roux, N. and Bengio, Y. (2008) · 2008
Earlier work this paper cites.
Training restricted Boltzmann machines using approximations to the likelihood gradient
Tieleman, T. (2008) · 2008
Earlier work this paper cites.

Similar

Deep Boltzmann machines
Salakhutdinov, R. and Hinton, G. (2009) · 2009
Cited alongside, same era.
Empirical risk minimization of graphical model parameters given approximate inference, decoding, and model structure
Stoyanov, V., Ropson, A., and Eisner, J. (2011) · 2011
Cited alongside, same era.
Layer-wise learning of deep generative models
Original
Arnold, L. and Ollivier, Y. (2012) · 2012
Cited alongside, same era.

Then

Improving neural networks by preventing co-adaptation of feature detectors
Original
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2012) · 2012
Later among the works it cites.
Learning feature hierarchies with cented deep Boltzmann machines
Original
Montavon, G. and Müller, K.-R. (2012) · 2012
Later among the works it cites.
Scaling up spike-and-slab models for unsupervised feature learning
Goodfellow, I. J., Courville, A., and Bengio, Y. (2013) · 2013
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…