Fetching the paper…
Reading the bibliography…
Deep learning has been immensely successful at a variety of tasks, ranging from classification to AI.
A.C. Anderson, Amorphous Solids: Low Temperature Properties , edited by W. A. Phillips, Topics in Current Physics, Vol. 24 (Springer, Berlin, 1981)
1981
Earlier work this paper cites.
Marc Mézard, Giorgio Parisi, and Miguel Virasoro, Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications , Vol. 9 (World Scientific Publishing Company, 1987)
1987
Earlier work this paper cites.
Elizabeth Gardner, “The space of interactions in neural network models,” Journal of physics A: Mathematical and general 21
1988
Earlier work this paper cites.
Eric B Baum, “On the capabilities of multilayer perceptrons,” Journal of complexity 4
1988
Earlier work this paper cites.
Yann LeCun, Yoshua Bengio, et al. , “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks 3361
1995
Earlier work this paper cites.
Rémi Monasson and Riccardo Zecchina, “Weight space structure and internal representations: a direct approach to learning and generalization in multilayer neural networks,” Physical review letters 75
1995
Earlier work this paper cites.
Alexei V. Tkachenko and Thomas A. Witten, “Stress propagation through frictionless granular material,” Phys. Rev. E 60
1999
Earlier work this paper cites.
Corey S. O’Hern, Leonardo E. Silbert, Andrea J. Liu, and Sidney R. Nagel, “Jamming at zero temperature and zero applied stress: The epitome of disorder,” Phys. Rev. E 68
2003
Earlier work this paper cites.
Aleksandar Donev, Ibrahim Cisse, David Sachs, Evan A. Variano, Frank H. Stillinger, Robert Connelly, Salvatore Torquato, and P. M. Chaikin, “Improving the density of jammed disordered packings using ellipsoids,” Science 303
2004
Earlier work this paper cites.
M. Wyart, “On the rigidity of amorphous solids,” Annales de Phys 30
2005
Earlier work this paper cites.
Matthieu Wyart, Leonardo E Silbert, Sidney R Nagel, and Thomas A Witten, “Effects of compression on the vibrational modes of marginally jammed solids,” Physical Review E 72
2005
Earlier work this paper cites.
L. E. Silbert, A. J. Liu, and S. R. Nagel, “Vibrations and diverging length scales near the unjamming transition,” Phys. Rev. Lett. 95
2005
Earlier work this paper cites.
Florent Krzakala and Jorge Kurchan, “Landscape analysis of constraint satisfaction problems,” Physical Review E 76
2007
Earlier work this paper cites.
Lenka Zdeborová and Florent Krzakala, “Phase transitions in the coloring of random graphs,” Physical Review E 76
2007
Earlier work this paper cites.
Mitch Mailman, Carl F. Schreck, Corey S. O’Hern, and Bulbul Chakraborty, “Jamming in systems composed of frictionless ellipse-shaped particles,” Phys. Rev. Lett. 102
2009
Earlier work this paper cites.
Z. Zeravcic, N. Xu, A. J. Liu, S. R. Nagel, and W. van Saarloos, “Excitations of ellipsoid packings near jamming,” Europhys. Lett. 87
2009
Earlier work this paper cites.
Andrea J. Liu, Sidney R. Nagel, W Saarloos, and Matthieu Wyart, Dynamical Heterogeneities in Glasses, Colloids, and Granular Media (2010)
2010
Earlier work this paper cites.
Ludovic Berthier and Giulio Biroli, “Theoretical perspective on the glass transition and amorphous materials,” Reviews of Modern Physics 83
2011
Earlier work this paper cites.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems (2012) pp. 1097–1105
2012
Earlier work this paper cites.
Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. , “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal processing magazine 29
2012
Earlier work this paper cites.
Matthieu Wyart, “Marginal stability constrains force and pair distributions at random close packing,” Phys. Rev. Lett. 109
2012
Earlier work this paper cites.
E. Lerner, G. Düring, and M. Wyart, “Toward a microscopic description of flow near the jamming threshold,” EPL (Europhysics Letters) 99
2012
Earlier work this paper cites.
Patrick Charbonneau, Eric I. Corwin, Giorgio Parisi, and Francesco Zamponi, “Universal microstructure and mechanical stability of jammed packings,” Physical Review Letters 109
2012
Earlier work this paper cites.
Gustavo Düring, Edan Lerner, and Matthieu Wyart, “Phonon gap and localization lengths in floppy materials,” Soft Matter 9
2013
Earlier work this paper cites.
Edan Lerner, Gustavo During, and Matthieu Wyart, “Low-energy non-linear excitations in sphere packings,” Soft Matter 9
2013
Cited alongside, same era.
Guido F Montufar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio, “On the number of linear regions of deep neural networks,” in Advances in neural information processing systems (2014) pp. 2924–2932
2014
Cited alongside, same era.
Monica Bianchini and Franco Scarselli, “On the complexity of neural network classifiers: A comparison between shallow and deep architectures,” IEEE transactions on neural networks and learning systems 25
2014
Cited alongside, same era.
A similar decomposition has been used in Pennington and Bahri 2017 ; Martens 2014 ; Sagun et al. 2017b
2014
Cited alongside, same era.
Eric DeGiuli, Adrien Laversanne-Finot, Gustavo Alberto Düring, Edan Lerner, and Matthieu Wyart, “Effects of coordination and pressure on sound attenuation, boson peak and elasticity in amorphous solids,” Soft Matter 10
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. , “Mastering the game of go without human knowledge,” Nature 550
2017
Later among the works it cites.
Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein, “On the expressive power of deep neural networks,” in Proceedings of the 34th International Conference on Machine Learning , Proceedings of Machine Learning Research, Vol. 70, edited by Doina Precup and Yee Whye Teh (PMLR, International Convention Centre, Sydney, Australia, 2017) pp. 2847–2854
2017
Later among the works it cites.
Holden Lee, Rong Ge, Tengyu Ma, Andrej Risteski, and Sanjeev Arora, “On the ability of neural nets to express distributions,” in Proceedings of the 2017 Conference on Learning Theory , Proceedings of Machine Learning Research, Vol. 65, edited by Satyen Kale and Ohad Shamir (PMLR, Amsterdam, Netherlands, 2017) pp. 1271–1296
2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2014
Cited alongside, same era.
Patrick Charbonneau, Jorge Kurchan, Giorgio Parisi, Pierfrancesco Urbani, and Francesco Zamponi, “Exact theory of dense amorphous hard spheres in high dimension. iii. the full replica symmetry breaking solution,” Journal of Statistical Mechanics: Theory and Experiment 2014
2014
Cited alongside, same era.
Andrew M Saxe, James L McClelland, and Surya Ganguli, “Exact solutions to the nonlinear dynamics of learning in deep linear neural networks,” International Conference on Learning Representations (2014)
2014
Cited alongside, same era.
2014
Cited alongside, same era.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, “Deep learning,” Nature 521
2015
Cited alongside, same era.
Sergey Ioffe and Christian Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International conference on machine learning (2015) pp. 448–456
2015
Cited alongside, same era.
Anna Choromanska, Mikael Henaff, Michael Mathieu, Gérard Ben Arous, and Yann LeCun, “The loss surfaces of multilayer networks,” in Artificial Intelligence and Statistics (2015) pp. 192–204
2015
Cited alongside, same era.
Markus Müller and Matthieu Wyart, “Marginal stability in structural, spin, and electron glasses,” Annual Review of Condensed Matter Physics 6
2015
Cited alongside, same era.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals, “Understanding deep learning requires rethinking generalization,” International Conference on Learning Representations (2017)
2017
Later among the works it cites.
C Daniel Freeman and Joan Bruna, “Topology and geometry of deep rectified network optimization landscapes,” International Conference on Learning Representations (2017)
2017
Later among the works it cites.
Elad Hoffer, Itay Hubara, and Daniel Soudry, “Train longer, generalize better: closing the generalization gap in large batch training of neural networks,” in Advances in Neural Information Processing Systems (2017) pp. 1729–1739
2017
Later among the works it cites.
Andrew J Ballard, Ritankar Das, Stefano Martiniani, Dhagash Mehta, Levent Sagun, Jacob D Stevenson, and David J Wales, “Energy landscapes for machine learning,” Physical Chemistry Chemical Physics (2017)
2017
Later among the works it cites.
Silvio Franz, Giorgio Parisi, Maxime Sevelev, Pierfrancesco Urbani, and Francesco Zamponi, “Universality of the sat-unsat (jamming) threshold in non-convex continuous constraint satisfaction problems,” SciPost Physics 2
2017
Later among the works it cites.
Silvio Franz and Stefano Spigler, “Mean-field avalanches in jammed spheres,” Physical Review E 95
2017
Later among the works it cites.
Xavier Gastaldi, “Shake-shake regularization of 3-branch residual networks,” International Conference on Learning Representations (2017)
2017
Later among the works it cites.
2017
Later among the works it cites.
2017
Later among the works it cites.
Jeffrey Pennington and Yasaman Bahri, “Geometry of neural network loss surfaces via random matrix theory,” in International Conference on Machine Learning (2017) pp. 2798–2806
2017
Later among the works it cites.
2018
Closest in time.
2018
Closest in time.
Marco Baity-Jesi, Levent Sagun, Mario Geiger, Stefano Spigler, Gerard Ben Arous, Chiara Cammarota, Yann LeCun, Matthieu Wyart, and Giulio Biroli, “Comparing dynamics: Deep neural networks versus glassy systems,” in Proceedings of the 35th International Conference on Machine Learning , Proceedings of Machine Learning Research, Vol. 80, edited by Jennifer Dy and Andreas Krause (PMLR, Stockholmsmässan, Stockholm Sweden, 2018) pp. 314–323
2018
Closest in time.
2018
Closest in time.
2018
Closest in time.
2018
Closest in time.
2018
Closest in time.
This transition influences the generalization properties of deep networks, too. This has been observed, for instance, in Advani and Saxe 2017 ; Spigler et al. 2018 , and studied by the authors in Geiger et al. 2019 (preprint)
2019
Closest in time.
2019
Closest in time.