Fetching the paper…

Reading the bibliography…

2012

On Training Deep Boltzmann Machines

Desjardins, Guillaume, Courville, Aaron, Bengio, Yoshua

Original
PDF
Post on X

Understand

The deep Boltzmann machine (DBM) has been an important development in the quest for powerful "deep" probabilistic models.

To date, simultaneous or joint training of all layers of the DBM has been largely unsuccessful with existing training methods.
We introduce a simple regularization scheme that encourages the weight vectors associated with each hidden unit to have similar norms.
We demonstrate that this regularization can be easily combined with standard stochastic maximum likelihood to yield an effective training strategy for the simultaneous training of all layers of the deep Boltzmann machine.

On Training Deep Boltzmann Machines · Around

Built on

Gradient based learning applied to document recognition
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998) · 1998
Earlier work this paper cites.
On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates
Younes, L. (1999) · 1999
Earlier work this paper cites.
Training products of experts by minimizing contrastive divergence
Hinton, G. E. (2000) · 2000
Earlier work this paper cites.
Annealed importance sampling
Neal, R. M. (2001) · 2001
Earlier work this paper cites.

Similar

A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., and Teh, Y. (2006) · 2006
Cited alongside, same era.
Sparse deep belief net model for visual area V2
Lee, H., Ekanadham, C., and Ng, A. (2008) · 2008
Cited alongside, same era.
On the quantitative analysis of deep belief networks
Salakhutdinov, R. and Murray, I. (2008) · 2008
Cited alongside, same era.

Then

Training restricted Boltzmann machines using approximations to the likelihood gradient
Tieleman, T. (2008) · 2008
Later among the works it cites.
Deep Boltzmann machines
Salakhutdinov, R. and Hinton, G. (2009) · 2009
Later among the works it cites.
Theano: a CPU and GPU math expression compiler
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., and Bengio, Y. (2010) · 2010
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…