Fetching the paper…
Reading the bibliography…
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities.
Solvable Model of a Spin-Glass
D. Sherrington and S. Kirkpatrick · 1975
Earlier work this paper cites.
Storing infinite numbers of patterns in a spin-glass model of neural networks
D. J. Amit, H. Gutfreund, and H. Sompolinsky · 1985
Earlier work this paper cites.
Spin Glass Theory and Beyond
M. Mézard, G. Parisi, and M. Virasoro · 1987
Earlier work this paper cites.
Spin Glass Theory and Beyond
M. Mézard, G. Parisi, and M. Virasoro · 1987
Earlier work this paper cites.
The space of interactions in neural network models
E. Gardner · 1988
Earlier work this paper cites.
Optimal storage properties of neural network models
E. Gardner and B. Derrida · 1988
Earlier work this paper cites.
Three unfinished works on the optimal storage capacity of networks
E. Gardner and B. Derrida · 1989
Earlier work this paper cites.
The space of interactions in neural networks: Gardner’s computation with the cavity method
M. Mézard · 1989
Earlier work this paper cites.
The space of interactions in neural networks: Gardner’s computation with the cavity method
M. Mézard · 1989
Earlier work this paper cites.
Statistical mechanics of learning from examples
H. S. Seung, H. Sompolinsky, and N. Tishby · 1992
Earlier work this paper cites.
Statistical mechanics of learning from examples
H. S. Seung, H. Sompolinsky, and N. Tishby · 1992
Earlier work this paper cites.
Replica field theory for deterministic models. II. A non-random spin glass with glassy behaviour
E. Marinari, G. Parisi, and F. Ritort · 1994
Earlier work this paper cites.
Mean-field equations for spin models with orthogonal interaction matrices
G. Parisi and M. Potters · 1995
Earlier work this paper cites.
The Information Bottleneck Method
N. Tishby, F. C. Pereira, and W. Bialek · 1999
Earlier work this paper cites.
Statistical Mechanics of Learning
A. Engel and C. Van den Broeck · 2001
Earlier work this paper cites.
Advanced mean field methods: Theory and practice
M. Opper and D. Saad · 2001
Earlier work this paper cites.
Statistical Physics of Spin Glasses and Information Processing: An Introduction
H. Nishimori · 2001
Earlier work this paper cites.
Statistical Mechanics of Learning
A. Engel and C. Van den Broeck · 2001
Earlier work this paper cites.
Tractable Approximations for Probabilistic Models: The Adaptive Thouless-Anderson-Palmer Mean Field Approach
M. Opper and O. Winther · 2001
Earlier work this paper cites.
A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors
T. Tanaka · 2002
Earlier work this paper cites.
Role of the interaction matrix in mean-field spin glass models
R. Cherrier, D. S. Dean, and A. Lefèvre · 2003
Earlier work this paper cites.
Spin Glasses: A Challenge for Mathematicians: Cavity and Mean Field Models
M. Talagrand · 2003
Earlier work this paper cites.
Estimating mutual information
A. Kraskov, H. Stögbauer, and P. Grassberger · 2004
Earlier work this paper cites.
Random Matrix Theory and Wireless Communications
A. M. Tulino and S. Verdú · 2004
Earlier work this paper cites.
Information bottleneck for Gaussian variables
G. Chechik, A. Globerson, N. Tishby, and Y. Weiss · 2005
Earlier work this paper cites.
Analysis of CDMA systems that are characterized by eigenvalue spectrum
K. Takeda, S. Uda, and Y. Kabashima · 2006
Earlier work this paper cites.
Efficient supervised learning in networks with binary synapses
C. Baldassi, A. Braunstein, N. Brunel, and R. Zecchina · 2007
Earlier work this paper cites.
Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels
Y. Kabashima · 2008
Earlier work this paper cites.
Vector Precoding for Wireless MIMO Systems and its Replica Analysis
R. R. Müller, D. Guo, and A. L. Moustakas · 2008
Earlier work this paper cites.
Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels
Y. Kabashima · 2008
Earlier work this paper cites.
Perceptron capacity revisited: classification ability for correlated patterns
T. Shinzato and Y. Kabashima · 2008
Cited alongside, same era.
Information, Physics, and Computation
M. Mézard and A. Montanari · 2009
Cited alongside, same era.
Message-passing algorithms for compressed sensing
D. Donoho, A. Maleki, and A. Montanari · 2009
Cited alongside, same era.
Information, Physics, and Computation
M. Mézard and A. Montanari · 2009
Cited alongside, same era.
Asymptotic analysis of MAP estimation via the replica method and compressed sensing
S. Rangan, V. Goyal, and A. K. Fletcher · 2009
Cited alongside, same era.
A typical reconstruction limit for compressed sensing based on Lp-norm minimization
Y. Kabashima, T. Wadayama, and T. Tanaka · 2009
Cited alongside, same era.
Multi-layer generalized linear estimation
A. Manoel, F. Krzakala, M. Mézard, and L. Zdeborová · 2017
Later among the works it cites.
Inference in Deep Networks in High Dimensions
A. K. Fletcher and S. Rangan · 2017
Later among the works it cites.
Additivity of Information in Multilayer Networks via Additive Gaussian Noise Transforms
G. Reeves · 2017
Later among the works it cites.
Phase Transitions, Optimal Errors and Optimality of Message-Passing in Generalized Linear Models
J. Barbier, F. Krzakala, N. Macris, L. Miolane, and L. Zdeborová · 2017
Later among the works it cites.
Vector approximate message passing
S. Rangan, P. Schniter, and A. K. Fletcher · 2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Learning from correlated patterns by simple perceptrons
T. Shinzato and Y. Kabashima · 2009
Cited alongside, same era.
Statistical Mechanics of Compressed Sensing
S. Ganguli and H. Sompolinsky · 2010
Cited alongside, same era.
Generalized approximate message passing for estimation with random linear mixing
S. Rangan · 2011
Cited alongside, same era.
Statistical-physics-based reconstruction in compressed sensing
F. Krzakala, M. Mézard, F. Sausset, Y. F. Sun, and L. Zdeborová · 2012
Cited alongside, same era.
Support Recovery With Sparsely Sampled Free Random Matrices
A. M. Tulino, G. Caire, S. Verdú, and S. Shamai (Shitz) · 2013
Cited alongside, same era.
Support Recovery With Sparsely Sampled Free Random Matrices
A. M. Tulino, G. Caire, S. Verdú, and S. Shamai (Shitz) · 2013
Cited alongside, same era.
J. Barbier and N. Macris · 2017
Later among the works it cites.
The Layered Structure of Tensor Estimation and its Mutual Information
J. Barbier, N. Macris, and L. Miolane · 2017
Later among the works it cites.
Harnessing neural networks: A random matrix approach
C. Louart and R. Couillet · 2017
Later among the works it cites.
Nonlinear random matrix theory for deep learning
J. Pennington and P. Worah · 2017
Later among the works it cites.
On the Expressive Power of Deep Neural Networks
M. Raghu, B. Poole, J. Kleinberg, S. Ganguli, and J. Sohl-Dickstein · 2017
Later among the works it cites.
Deep information propagation
S.S. Schoenholz, J. Gilmer, S. Ganguli, and J. Sohl-Dickstein · 2017
Later among the works it cites.
High-dimensional dynamics of generalization error in neural networks
M. Advani and A. Saxe · 2017
Later among the works it cites.
Deep variational information bottleneck
A. Alemi, I. Fischer, J. Dillon, and K. Murphy · 2017
Later among the works it cites.
Emergence of Invariance and Disentangling in Deep Representations
A. Achille and S. Soatto · 2017
Later among the works it cites.
Nonlinear Information Bottleneck
A. Kolchinsky, B. D. Tracey, and D. H. Wolpert · 2017
Later among the works it cites.
InfoVAE: Information Maximizing Variational Autoencoders
S. Zhao, J. Song, and S. Ermon · 2017
Later among the works it cites.
Estimating mixture entropy with pairwise distances
A. Kolchinsky and B. D. Tracey · 2017
Later among the works it cites.
Multi-layer generalized linear estimation
A. Manoel, F. Krzakala, M. Mézard, and L. Zdeborová · 2017
Later among the works it cites.
Inference in Deep Networks in High Dimensions
A. K. Fletcher and S. Rangan · 2017
Later among the works it cites.
Additivity of Information in Multilayer Networks via Additive Gaussian Noise Transforms
G. Reeves · 2017
Later among the works it cites.
Phase Transitions, Optimal Errors and Optimality of Message-Passing in Generalized Linear Models
J. Barbier, F. Krzakala, N. Macris, L. Miolane, and L. Zdeborová · 2017
Later among the works it cites.
The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference
J. Barbier and N. Macris · 2017
Later among the works it cites.
The Layered Structure of Tensor Estimation and its Mutual Information
J. Barbier, N. Macris, and L. Miolane · 2017
Later among the works it cites.
Opening the Black Box of Deep Neural Networks via Information
R. Shwartz-Ziv and N. Tishby · 2017
Later among the works it cites.
On the Information Bottleneck Theory of Deep Learning
A. M. Saxe, Y. Bansal, J. Dapello, M. Advani, A. Kolchinsky, B. D. Tracey, and D. D. Cox · 2018
Closest in time.
The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices
J. Barbier, N. Macris, A. Maillard, and F. Krzakala · 2018
Closest in time.
Information Dropout: Learning Optimal Representations Through Noisy Computation
A. Achille and S. Soatto · 2018
Closest in time.
MINE: Mutual Information Neural Estimation
M.I. Belghazi, A. Baratin, S. Rajeswar, S. Ozair, Y. Bengio, A. Courville, and R.D. Hjelm · 2018
Closest in time.
The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices
J. Barbier, N. Macris, A. Maillard, and F. Krzakala · 2018
Closest in time.