Fetching the paper…
Reading the bibliography…
Motivated by recent progress in using restricted Boltzmann machines as preprocessing algorithms for deep neural network, we revisit the mean-field equations (belief-propagation and TAP equations) in the best understood such machine, namely the Hopfield model of neural networks, and we explicit how they can be used as iterative message-passing algorithms, providing a fast method to compute the local polarizations of neurons.
M. V.A. and P. L.A., “Distribution of eigenvalues for some sets of random matrices,” Math. USSR Sb. 1
1967
Earlier work this paper cites.
D. Sherrington and S. Kirkpatrick, “Solvable model of a spin-glass,” Physical review letters 35
1975
Earlier work this paper cites.
D. J. Thouless, P. W. Anderson, and R. G. Palmer, “Solution of’solvable model of a spin glass’,” Philosophical Magazine 35
1977
Earlier work this paper cites.
J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” PNAS 79
1982
Earlier work this paper cites.
M. Mézard, G. Parisi, and M. A. Virasoro, Spin-Glass Theory and Beyond (World Scientific, Singapore, 1987)
1987
Earlier work this paper cites.
M. Mézard, “The space of interactions in neural networks: Gardner’s computation with the cavity method,” Journal of Physics A: Mathematical and General 22
1989
Earlier work this paper cites.
K. Nakanishi and H. Takayama, “Mean-field theory for a spin-glass model of neural networks: Tap free energy and the paramagnetic to spin-glass transition,” Journal of Physics A: Mathematical and General 30
1997
Earlier work this paper cites.
H. J. Kappen and F. Rodriguez, “Boltzmann machine learning using mean field theory and linear response correction,” Advances in neural information processing systems , 280–286 (1998)
1998
Earlier work this paper cites.
T. Tanaka, “Mean-field theory of boltzmann machine learning,” Physical Review E 58
1998
Earlier work this paper cites.
M. Shamir and H. Sompolinsky, “Thouless-anderson-palmer equations for neural networks,” Physical Review E 61
2000
Earlier work this paper cites.
Y. Kabashima and D. Saad, “The tap approach to intensive and extensive connectivity systems,” Advanced Mean Field Methods–Theory and Practice 6
2001
Earlier work this paper cites.
M. Opper and O. Winther, “Adaptive and self-averaging thouless-anderson-palmer mean-field theory for probabilistic modeling,” Physical Review E 64
2001
Earlier work this paper cites.
M. Welling and G. E. Hinton, “A new learning algorithm for mean field boltzmann machines,” in International Conference on Artificial Neural Networks (Springer, 2002) pp. 351–357
2002
Earlier work this paper cites.
Y. Kabashima, “A cdma multiuser detection algorithm on the basis of belief propagation,” Journal of Physics A: Mathematical and General 36
2003
Cited alongside, same era.
T. Tanaka and M. Okada, “Approximate belief propagation, density evolution, and statistical neurodynamics for cdma multiuser detection,” IEEE Transactions on Information Theory 51
2005
Cited alongside, same era.
A. Montanari and D. Tse, “Analysis of belief propagation for non-linear problems: The example of cdma (or: How to prove tanaka’s formula),” in 2006 IEEE Information Theory Workshop-ITW’06 Punta del Este (IEEE, 2006) pp. 160–164
2006
Cited alongside, same era.
D. Guo and C.-C. Wang, “Asymptotic mean-square optimality of belief propagation for sparse linear systems,” Information Theory Workshop, 2006. ITW ’06 Chengdu. , 194–198 (2006)
2006
Cited alongside, same era.
M. Bayati and A. Montanari, “The dynamics of message passing on dense graphs, with applications to compressed sensing,” IEEE Transactions on Information Theory 57
2011
Later among the works it cites.
S. Rangan, “Generalized approximate message passing for estimation with random linear mixing,” in IEEE International Symposium on Information Theory Proceedings (ISIT) (2011) pp. 2168 –2172
2011
Later among the works it cites.
J. T. Parker, V. Cevher, and P. Schniter, “Compressive sensing under matrix uncertainties: An approximate message passing approach,” in Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR) (2011) pp. 804–808
2011
Later among the works it cites.
P. Schniter, J. Parker, and V. Cevher, “Bilinear generalized approximate message passing (big-amp) for matrix recovery problem,” in Workshop on Information Theory and Applications (ITA), San Diego CA (2012)
2012
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
R. Salakhutdinov, A. Mnih, and G. Hinton, “Restricted boltzmann machines for collaborative filtering,” in Proceedings of the 24th international conference on Machine learning (ACM, 2007) pp. 791–798
2007
Cited alongside, same era.
C. Baldassi, A. Braunstein, N. Brunel, and R. Zecchina, “Efficient supervised learning in networks with binary synapses,” BMC Neuroscience 8
2007
Cited alongside, same era.
P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, “Extracting and composing robust features with denoising autoencoders,” in Proceedings of the 25th international conference on Machine learning (ACM, 2008) pp. 1096–1103
2008
Cited alongside, same era.
T. Tieleman, “Training restricted boltzmann machines using approximations to the likelihood gradient,” in Proceedings of the 25th international conference on Machine learning (ACM, 2008) pp. 1064–1071
2008
Cited alongside, same era.
T. Richardson and R. Urbanke, Modern Coding Theory (Cambridge University Press, 2008)
2008
Cited alongside, same era.
M. Mézard and A. Montanari, Information, Physics and Computation (Oxford University Press, Oxford, 2009)
2009
Cited alongside, same era.
D. L. Donoho, A. Maleki, and A. Montanari, “Message-passing algorithms for compressed sensing,” Proc. Natl. Acad. Sci. 106
2009
Cited alongside, same era.
F. Krzakala and L. Zdeborová, “Hiding quiet solutions in random constraint satisfaction problems,” Phys. Rev. Lett. 102
2009
Cited alongside, same era.
E. Bolthausen, “An iterative construction of solutions of the tap equations for the sherrington–kirkpatrick model,” Communications in Mathematical Physics 325
2014
Later among the works it cites.
Y. Deshpande and A. Montanari, “Sparse pca via covariance thresholding,” in Advances in Neural Information Processing Systems (2014) pp. 334–342
2014
Later among the works it cites.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521
2015
Later among the works it cites.
H. Huang and T. Toyoizumi, “Advanced mean-field theory of the restricted boltzmann machine,” Physical Review E 91
2015
Later among the works it cites.
M. Gabrié, E. W. Tramel, and F. Krzakala, “Training restricted boltzmann machine via the thouless-anderson-palmer free energy,” in Advances in Neural Information Processing Systems (2015) pp. 640–648
2015
Later among the works it cites.
2015
Later among the works it cites.
C. Baldassi and A. Braunstein, “A max-sum algorithm for training discrete neural networks,” Journal of Statistical Mechanics: Theory and Experiment 2015
2015
Later among the works it cites.
T. Lesieur, F. Krzakala, and L. Zdeborov, “Mmse of probabilistic low-rank matrix estimation: Universality with respect to the output channel,” in 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton) (IEEE, 2015) pp. 680–687
2015
Later among the works it cites.