Fetching the paper…
Reading the bibliography…
In these pedagogic notes I review the statistical mechanics approach to neural networks, focusing on the paradigmatic example of the perceptron architecture with binary an continuous weights, in the classification setting.
Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition ,
T. M. Cover, · 1965
Earlier work this paper cites.
Solvable model of a spin-glass ,
D. Sherrington and S. Kirkpatrick, · 1975
Earlier work this paper cites.
A sequence of approximated solutions to the s-k model for spin glasses ,
G. Parisi, · 1980
Earlier work this paper cites.
Maximum storage capacity in neural networks ,
E. Gardner, · 1987
Earlier work this paper cites.
Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications , vol. 9,
M. Mézard, G. Parisi and M. Virasoro, · 1987
Earlier work this paper cites.
The space of interactions in neural network models ,
E. Gardner, · 1988
Earlier work this paper cites.
Optimal storage properties of neural network models ,
E. Gardner and B. Derrida, · 1988
Earlier work this paper cites.
Three unfinished works on the optimal storage capacity of networks ,
E. Gardner and B. Derrida, · 1989
Earlier work this paper cites.
Storage capacity of memory networks with binary couplings ,
W. Krauth and M. Mézard, · 1989
Earlier work this paper cites.
First-order transition to perfect generalization in a neural network with binary synapses ,
G. Györgyi, · 1990
Earlier work this paper cites.
Recipes for metastable states in spin glasses ,
S. Franz and G. Parisi, · 1995
Earlier work this paper cites.
Statistical mechanics of learning ,
A. Engel and C. Van den Broeck, · 2001
Earlier work this paper cites.
Learning by message passing in networks of discrete synapses ,
A. Braunstein and R. Zecchina, · 2006
Earlier work this paper cites.
Efficient supervised learning in networks with binary synapses ,
C. Baldassi, A. Braunstein, N. Brunel and R. Zecchina, · 2007
Earlier work this paper cites.
Origin of the computational hardness for learning with binary synapses ,
H. Huang and Y. Kabashima, · 2014
Earlier work this paper cites.
Qualitatively characterizing neural network optimization problems ,
I. J. Goodfellow, O. Vinyals and A. M. Saxe, · 2014
Earlier work this paper cites.
Subdominant dense clusters allow for simple learning and high computational performance in neural networks with discrete synapses ,
C. Baldassi, A. Ingrosso, C. Lucibello, L. Saglietti and R. Zecchina, · 2015
Earlier work this paper cites.
Eigenvalues of the hessian in deep learning: Singularity and beyond ,
L. Sagun, L. Bottou and Y. LeCun, · 2016
Earlier work this paper cites.
Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes ,
C. Baldassi, C. Borgs, J. T. Chayes, A. Ingrosso, C. Lucibello, L. Saglietti and R. Zecchina, · 2016
Cited alongside, same era.
Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems ,
S. Franz, G. Parisi, M. Sevelev, P. Urbani and F. Zamponi, · 2017
Cited alongside, same era.
Empirical analysis of the hessian of over-parametrized neural networks ,
L. Sagun, U. Evci, V. U. Guney, Y. Dauphin and L. Bottou, · 2017
Cited alongside, same era.
Essentially no barriers in neural network energy landscape ,
F. Draxler, K. Veschgini, M. Salmhofer and F. Hamprecht, · 2018
Cited alongside, same era.
Visualizing the loss landscape of neural nets ,
H. Li, Z. Xu, G. Taylor, C. Studer and T. Goldstein, · 2018
Cited alongside, same era.
Proof of the contiguity conjecture and lognormal limit for the symmetric perceptron ,
E. Abbe, S. Li and A. Sly, · 2021
Later among the works it cites.
Frozen 1-rsb structure of the symmetric ising perceptron ,
W. Perkins and C. Xu, · 2021
Later among the works it cites.
The overlap gap property: A topological barrier to optimizing over random structures ,
D. Gamarnik, · 2021
Later among the works it cites.
Optimization of mean-field spin glasses ,
A. El Alaoui, A. Montanari and M. Sellke, · 2021
Later among the works it cites.
Binary perceptron: efficient algorithms can find solutions in a rare well-connected cluster ,
E. Abbe, S. Li and A. Sly, · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Loss surfaces, mode connectivity, and fast ensembling of dnns ,
T. Garipov, P. Izmailov, D. Podoprikhin, D. P. Vetrov and A. G. Wilson, · 2018
Cited alongside, same era.
Capacity lower bound for the ising perceptron ,
J. Ding and N. Sun, · 2019
Cited alongside, same era.
Comparing dynamics: deep neural networks versus glassy systems ,
M. Baity-Jesi, L. Sagun, M. Geiger, S. Spigler, G. B. Arous, C. Cammarota, Y. LeCun, M. Wyart and G. Biroli, · 2019
Cited alongside, same era.
Biased landscapes for random constraint satisfaction problems ,
L. Budzynski, F. Ricci-Tersenghi and G. Semerjian, · 2019
Cited alongside, same era.
Properties of the geometry of solutions and capacity of multilayer neural networks with rectified linear unit activations ,
C. Baldassi, E. M. Malatesta and R. Zecchina, · 2019
Cited alongside, same era.
Shaping the learning landscape in neural networks around wide flat minima ,
C. Baldassi, F. Pittorino and R. Zecchina, · 2020
Cited alongside, same era.
Wide flat minima and optimal generalization in classifying high-dimensional gaussian mixtures ,
C. Baldassi, E. M. Malatesta, M. Negri and R. Zecchina, · 2020
Cited alongside, same era.
A. G. Cavaliere, T. Lesieur and F. Ricci-Tersenghi, · 2021
Later among the works it cites.
The role of permutation invariance in linear mode connectivity of neural networks ,
R. Entezari, H. Sedghi, O. Saukh and B. Neyshabur, · 2022
Later among the works it cites.
Deep networks on toroids: Removing symmetries reveals the structure of flat regions in the landscape geometry ,
F. Pittorino, A. Ferraro, G. Perugini, C. Feinauer, C. Baldassi and R. Zecchina, · 2022
Later among the works it cites.
Anomalous diffusion dynamics of learning in deep neural networks ,
G. Chen, C. K. Qu and P. Gong, · 2022
Later among the works it cites.
Algorithms and barriers in the symmetric binary perceptron model ,
D. Gamarnik, E. C. Kızıldağ, W. Perkins and C. Xu, · 2022
Later among the works it cites.
Learning through atypical phase transitions in overparameterized neural networks ,
C. Baldassi, C. Lauditi, E. M. Malatesta, R. Pacelli, G. Perugini and R. Zecchina, · 2022
Later among the works it cites.
Algorithmic pure states for the negative spherical perceptron ,
A. El Alaoui and M. Sellke, · 2022
Later among the works it cites.
What can linear interpolation of neural network loss landscapes tell us? ,
T. J. Vlaar and J. Frankle, · 2022
Later among the works it cites.
Typical and atypical solutions in nonconvex neural networks with discrete and continuous weights ,
C. Baldassi, E. M. Malatesta, G. Perugini and R. Zecchina, · 2023
Closest in time.
Geometric barriers for stable and online algorithms for discrepancy minimization ,
D. Gamarnik, E. C. Kizildağ, W. Perkins and C. Xu, · 2023
Closest in time.
Plateau in monotonic linear interpolation — a ”biased” view of loss landscape for deep networks ,
X. Wang, A. N. Wang, M. Zhou and R. Ge, · 2023
Closest in time.
The star-shaped space of solutions of the spherical negative perceptron ,
B. L. Annesi, C. Lauditi, C. Lucibello, E. M. Malatesta, G. Perugini, F. Pittorino and L. Saglietti, · 2023
Closest in time.