Fetching the paper…
Reading the bibliography…
We demonstrate that conventional artificial deep neural networks operating near the phase boundary of the signal propagation dynamics, also known as the edge of chaos, exhibit universal scaling laws of absorbing phase transitions in non-equilibrium statistical mechanics.
A. M. Turing, I.―COMPUTING MACHINERY AND INTELLIGENCE, Mind LIX
1950
Earlier work this paper cites.
S. R. Broadbent and J. M. Hammersley, Percolation processes: I. crystals and mazes, Mathematical Proceedings of the Cambridge Philosophical Society 53
1957
Earlier work this paper cites.
A. E. Ferdinand and M. E. Fisher, Bounded and inhomogeneous ising models. i. specific-heat anomaly of a finite lattice, Phys. Rev. 185
1969
Earlier work this paper cites.
T. E. Harris, Contact interactions on a lattice, The Annals of Probability 2
1974
Earlier work this paper cites.
V. Privman and M. E. Fisher, Universal critical amplitudes in finite-size scaling, Phys. Rev. B 30
1984
Earlier work this paper cites.
K. Kaneko, Period-Doubling of Kink-Antikink Patterns, Quasiperiodicity in Antiferro-Like Structures and Spatial Intermittency in Coupled Logistic Lattice: Towards a Prelude of a “Field Theory of Chaos”, Progress of Theoretical Physics 72
1984
Earlier work this paper cites.
P. Bak, C. Tang, and K. Wiesenfeld, Self-organized criticality: An explanation of the 1/f noise, Phys. Rev. Lett. 59
1987
Earlier work this paper cites.
H. K. Janssen, B. Schaub, and B. Schmittmann, New universal short-time scaling behaviour of critical relaxation processes, Zeitschrift für Physik B Condensed Matter 73
1989
Earlier work this paper cites.
C. G. Langton, Computation at the edge of chaos: Phase transitions and emergent computation, Physica D: Nonlinear Phenomena 42
1990
Earlier work this paper cites.
R. M. Neal, Priors for infinite networks, in Bayesian Learning for Neural Networks (Springer New York, New York, NY, 1996) pp. 29–53
1996
Earlier work this paper cites.
C. Williams, Computing with Infinite Networks, in Advances in Neural Information Processing Systems , Vol. 9, edited by M. Mozer, M. Jordan, and T. Petsche (MIT Press, 1996)
1996
Earlier work this paper cites.
R. Coutinho and B. Fernandez, Extended symbolic dynamics in bistable CML: Existence and stability of fronts, Physica D: Nonlinear Phenomena 108
1997
Earlier work this paper cites.
B. Fernandez and L. Raymond, Propagating fronts in a bistable coupled map lattice, Journal of Statistical Physics 86
1997
Earlier work this paper cites.
G. Grinstein and M. A. Muñoz, The Statistical Mechanics of Absorbing States, in Fourth Granada Lectures in Computational Physics , edited by P. L. Garrido and J. Marro (Springer Berlin Heidelberg, Berlin, Heidelberg, 1997) pp. 223–270
1997
Earlier work this paper cites.
C. A. Voigt and R. M. Ziff, Epidemic analysis of the second-order transition in the ziff-gulari-barshad surface-reaction model, Phys. Rev. E 56
1997
Earlier work this paper cites.
R. Dickman, A. Vespignani, and S. Zapperi, Self-organized criticality as an absorbing-state phase transition, Phys. Rev. E 57
1998
Earlier work this paper cites.
I. Jensen, Low-density series expansions for directed percolation: I. a new efficient algorithm with applications to the square lattice, Journal of Physics A: Mathematical and General 32
1999
Earlier work this paper cites.
H. Hinrichsen, Non-equilibrium critical phenomena and phase transitions into absorbing states, Advances in Physics 49
2000
Earlier work this paper cites.
A. Guttmann, Indicators of solvability for lattice models, Discrete Mathematics 217
2000
Earlier work this paper cites.
B. Zheng, Monte carlo simulations and numerical solutions of short-time critical dynamics, Physica A: Statistical Mechanics and its Applications 283
2000
Earlier work this paper cites.
2001
Earlier work this paper cites.
2003
Earlier work this paper cites.
N. Bertschinger, T. Natschläger, and R. Legenstein, At the edge of chaos: Real-time computations and self-organized criticality in recurrent neural networks, in Advances in Neural Information Processing Systems , Vol. 17, edited by L. Saul, Y. Weiss, and L. Bottou (MIT Press, 2004)
2004
Earlier work this paper cites.
J. Bickle, Reducing mind to molecular pathways: explicating the reductionism implicit in current cellular and molecular neuroscience, Synthese 151
2006
Earlier work this paper cites.
J. E. Kolassa, Edgeworth series, in Series Approximation Methods in Statistics (Springer New York, New York, NY, 2006) pp. 31–62
2006
Earlier work this paper cites.
M. Henkel, H. Hinrichsen, and S. Lübeck, Non-Equilibrium Phase Transitions , 1st ed. (Springer, 2008)
2008
Earlier work this paper cites.
Y. Cho and L. Saul, Kernel Methods for Deep Learning, in Advances in Neural Information Processing Systems , Vol. 22, edited by Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta (Curran Associates, Inc., 2009)
2009
Earlier work this paper cites.
N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, Front-End Factor Analysis for Speaker Verification, IEEE Transactions on Audio, Speech, and Language Processing 19
2011
Earlier work this paper cites.
N. Friedman, S. Ito, B. A. W. Brinkman, M. Shimono, R. E. L. DeVille, K. A. Dahmen, J. M. Beggs, and T. C. Butler, Universal Critical Dynamics in High Resolution Neuronal Avalanche Data, Phys. Rev. Lett. 108
2012
Cited alongside, same era.
G. Lan, P. Sartori, S. Neumann, V. Sourjik, and Y. Tu, The energy–speed–accuracy trade-off in sensory adaptation, Nature Physics 8
2012
Cited alongside, same era.
S. Zeraati, F. H. Jafarpour, and H. Hinrichsen, Entropy production of nonequilibrium steady states with irreversible transitions, Journal of Statistical Mechanics: Theory and Experiment 2012
2012
Cited alongside, same era.
J. Wang, Z. Zhou, Q. Liu, T. M. Garoni, and Y. Deng, High-precision monte carlo study of directed percolation in ( d + 1 d+1 ) dimensions, Phys. Rev. E 88
2013
Cited alongside, same era.
L. Chizat, E. Oyallon, and F. Bach, On Lazy Training in Differentiable Programming, in Advances in Neural Information Processing Systems , Vol. 32, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Curran Associates, Inc., 2019)
2019
Later among the works it cites.
A. J. Fontenele, N. A. P. de Vasconcelos, T. Feliciano, L. A. A. Aguiar, C. Soares-Cunha, B. Coimbra, L. Dalla Porta, S. Ribeiro, A. J. a. Rodrigues, N. Sousa, P. V. Carelli, and M. Copelli, Criticality between Cortical States, Phys. Rev. Lett. 122
2019
Later among the works it cites.
K. Harada and N. Kawashima, Entropy governed by the absorbing state of directed percolation, Phys. Rev. Lett. 123
2019
Later among the works it cites.
F. Stahlberg, Neural Machine Translation: A Review, Journal of Artificial Intelligence Research 69
2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2013
Cited alongside, same era.
J. Hesse and T. Gross, Self-organized criticality as a fundamental property of neural systems, Frontiers in Systems Neuroscience 8
2014
Cited alongside, same era.
W. L. Shew, W. P. Clawson, J. Pobst, Y. Karimipanah, N. C. Wright, and R. Wessel, Adaptation to sensory input tunes visual cortex to criticality, Nature Physics 11
2015
Cited alongside, same era.
R. Rao and L. Peliti, Thermodynamics of accuracy in kinetic proofreading: dissipation and efficiency trade-offs, Journal of Statistical Mechanics: Theory and Experiment 2015
2015
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun, Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, in Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2015)
2015
Cited alongside, same era.
2015
Cited alongside, same era.
D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, Mastering the game of go with deep neural networks and tree search, Nature 529
2016
Cited alongside, same era.
B. Poole, S. Lahiri, M. Raghu, J. Sohl-Dickstein, and S. Ganguli, Exponential expressivity in deep neural networks through transient chaos, in Advances in Neural Information Processing Systems , Vol. 29, edited by D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Curran Associates, Inc., 2016)
2016
Cited alongside, same era.
L. Xiao, J. Pennington, and S. Schoenholz, Disentangling trainability and generalization in deep neural networks, in Proceedings of the 37th International Conference on Machine Learning , Proceedings of Machine Learning Research, Vol. 119, edited by H. D. III and A. Singh (PMLR, 2020) pp. 10462–10472
2020
Later among the works it cites.
S. Peluchetti and S. Favaro, Infinitely deep neural networks as diffusion processes, in Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , Proceedings of Machine Learning Research, Vol. 108, edited by S. Chiappa and R. Calandra (PMLR, 2020) pp. 1126–1136
2020
Later among the works it cites.
S. Yaida, Non-Gaussian processes and neural networks at finite widths, in Proceedings of The First Mathematical and Scientific Machine Learning Conference , Proceedings of Machine Learning Research, Vol. 107, edited by J. Lu and R. Ward (PMLR, 2020) pp. 165–192
2020
Later among the works it cites.
J. Huang and H.-T. Yau, Dynamics of Deep Neural Networks and Neural Tangent Hierarchy, in Proceedings of the 37th International Conference on Machine Learning , Proceedings of Machine Learning Research, Vol. 119, edited by H. D. III and A. Singh (PMLR, 2020) pp. 4542–4551
2020
Later among the works it cites.
Z. Chen, Y. Cao, Q. Gu, and T. Zhang, A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks, in Advances in Neural Information Processing Systems , Vol. 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (Curran Associates, Inc., 2020) pp. 13363–13373
2020
Later among the works it cites.
L. Lu, Y. Shin, Y. Su, and G. Em Karniadakis, Dying ReLU and Initialization: Theory and Numerical Examples, Communications in Computational Physics 28
2020
Later among the works it cites.
E. R. Kandel, J. D. Koester, S. H. Mack, and S. A. Siegelbaum, Principles of Neural Science, Sixth Edition (McGraw Hill, New York, 2021)
2021
Later among the works it cites.
M. Girardi-Schappo, Brain criticality beyond avalanches: open problems and how to approach them, Journal of Physics: Complexity 2
2021
Later among the works it cites.
C. Gros, A devil’s advocate view on ‘self-organized’ brain criticality, Journal of Physics: Complexity 2
2021
Later among the works it cites.
P. Ju, X. Lin, and N. Shroff, On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models, in Proceedings of the 38th International Conference on Machine Learning , Proceedings of Machine Learning Research, Vol. 139, edited by M. Meila and T. Zhang (PMLR, 2021) pp. 5137–5147
2021
Later among the works it cites.
J. M. Beggs, Addressing skepticism of the critical brain hypothesis, Frontiers in Computational Neuroscience 16
2022
Later among the works it cites.
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, High-resolution image synthesis with latent diffusion models, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022) pp. 10684–10695
2022
Later among the works it cites.
S. Hayou, A. Doucet, and J. Rousseau, The curse of depth in kernel regime, in Proceedings on ”I (Still) Can’t Believe It’s Not Better!” at NeurIPS 2021 Workshops , Proceedings of Machine Learning Research, Vol. 163, edited by M. F. Pradier, A. Schein, S. Hyland, F. J. R. Ruiz, and J. Z. Forde (PMLR, 2022) pp. 41–47
2022
Later among the works it cites.
2022
Later among the works it cites.
K. T. Grosvenor and R. Jefferson, The edge of chaos: quantum field theory and deep neural networks, SciPost Phys. 12
2022
Later among the works it cites.
E. Nichani, Y. Bai, and J. D. Lee, Identifying good directions to escape the ntk regime and efficiently learn low-degree plus sparse polynomials, in Advances in Neural Information Processing Systems , Vol. 35, edited by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Curran Associates, Inc., 2022) pp. 14568–14581
2022
Later among the works it cites.
S. R. Dubey, S. K. Singh, and B. B. Chaudhuri, Activation functions in deep learning: A comprehensive survey and benchmark, Neurocomputing 503
2022
Later among the works it cites.
C. Gervais, L.-P. Boucher, G. M. Villar, U. Lee, and C. Duclos, A scoping review for building a criticality-based conceptual framework of altered states of consciousness, Frontiers in Systems Neuroscience 17
2023
Closest in time.
2024
Closest in time.
B. Hanin, Random fully connected neural networks as perturbatively solvable hierarchies, Journal of Machine Learning Research 25
2024
Closest in time.
Y. Xu, A. Schneider, R. Wessel, and K. B. Hengen, Sleep restores an optimal computational regime in cortical networks, Nature Neuroscience 27
2024
Closest in time.
S. Takasu and T. Aoyagi, Suppression of chaos in a partially driven recurrent neural network, Phys. Rev. Res. 6
2024
Closest in time.