Fetching the paper…
Reading the bibliography…
The biological implausibility of backpropagation (BP) has motivated many alternative, brain-inspired algorithms that attempt to rely only on local information, such as predictive coding (PC) and equilibrium propagation.
Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgleichungen (mit einer anwendung auf die theorie der hohlraumstrahlung)
H. Weyl · 1912
Earlier work this paper cites.
Distribution of eigenvalues for some sets of random matrices
V. A. Marchenko and L. A. Pastur · 1967
Earlier work this paper cites.
Learning representations by back-propagating errors
D. E. Rumelhart, G. E. Hinton, and R. J. Williams · 1986
Earlier work this paper cites.
Characteristic vectors of bordered matrices with infinite dimensions i
E. P. Wigner · 1993
Earlier work this paper cites.
Efficient backprop
Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Müller · 2002
Earlier work this paper cites.
Convex optimization
S. Boyd and L. Vandenberghe · 2004
Earlier work this paper cites.
Comments on “a note on a three-term recurrence for a tridiagonal matrix”
D. K. Salkuyeh · 2006
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
X. Glorot and Y. Bengio · 2010
Earlier work this paper cites.
Matrix analysis
R. A. Horn and C. R. Johnson · 2012
Earlier work this paper cites.
Introductory lectures on convex optimization: A basic course
Y. Nesterov · 2013
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
A. M. Saxe, J. L. McClelland, and S. Ganguli · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
D. P. Kingma and J. Ba · 2014
Earlier work this paper cites.
The loss surfaces of multilayer networks
A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun · 2015
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
K. He, X. Zhang, S. Ren, and J. Sun · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
S. Ioffe · 2015
Earlier work this paper cites.
Deep learning
Y. LeCun, Y. Bengio, and G. Hinton · 2015
Earlier work this paper cites.
J. L. Ba, J. R. Kiros, and G. E. Hinton · 2016
Earlier work this paper cites.
Identity matters in deep learning
M. Hardt and T. Ma · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Random synaptic feedback weights support error backpropagation for deep learning
T. P. Lillicrap, D. Cownden, D. B. Tweed, and C. J. Akerman · 2016
Earlier work this paper cites.
The free energy principle for action and perception: A mathematical review
C. L. Buckley, C. S. Kim, S. McGregor, and A. K. Seth · 2017
Earlier work this paper cites.
Geometry of neural network loss surfaces via random matrix theory
J. Pennington and Y. Bahri · 2017
Earlier work this paper cites.
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
J. Pennington, S. Schoenholz, and S. Ganguli · 2017
Earlier work this paper cites.
Equilibrium propagation: Bridging the gap between energy-based models and backpropagation
B. Scellier and Y. Bengio · 2017
Earlier work this paper cites.
Structured random matrices
R. Van Handel · 2017
Earlier work this paper cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin · 2017
Cited alongside, same era.
An approximation of the error backpropagation algorithm in a predictive coding network with local hebbian synaptic plasticity
J. C. Whittington and R. Bogacz · 2017
Cited alongside, same era.
Neural tangent kernel: Convergence and generalization in neural networks
A. Jacot, F. Gabriel, and C. Hongler · 2018
Cited alongside, same era.
The emergence of spectral universality in deep networks
J. Pennington, S. Schoenholz, and S. Ganguli · 2018
Cited alongside, same era.
Dynamical isometry and a mean field theory of cnns: How to train 10,000-layer vanilla convolutional neural networks
L. Xiao, Y. Bahri, J. Sohl-Dickstein, S. Schoenholz, and J. Pennington · 2018
Cited alongside, same era.
On lazy training in differentiable programming
Learning on arbitrary graph topologies via predictive coding
T. Salvatori, L. Pinchetti, B. Millidge, Y. Song, T. Bao, R. Bogacz, and T. Lukasiewicz · 2022
Later among the works it cites.
Incremental predictive coding: A parallel and fully automatic learning algorithm
T. Salvatori, Y. Song, B. Millidge, Z. Xu, L. Sha, C. Emde, R. Bogacz, and T. Lukasiewicz · 2022
Later among the works it cites.
Inferring neural activity before plasticity: A foundation for learning beyond backpropagation
Y. Song, B. Millidge, T. Salvatori, T. Lukasiewicz, Z. Xu, and R. Bogacz · 2022
Later among the works it cites.
Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation
N. Zucchet and J. Sacramento · 2022
Later among the works it cites.
Depthwise hyperparameter transfer in residual networks: Dynamics and scaling limit
B. Bordelon, L. Noci, M. B. Li, B. Hanin, and C. Pehlevan · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
L. Chizat, E. Oyallon, and F. Bach · 2019
Cited alongside, same era.
Wide neural networks of any depth evolve as linear models under gradient descent
J. Lee, L. Xiao, S. Schoenholz, Y. Bahri, R. Novak, J. Sohl-Dickstein, and J. Pennington · 2019
Cited alongside, same era.
Beyond random matrix theory for deep networks
D. Granziol · 2020
Cited alongside, same era.
Backpropagation and the brain
T. P. Lillicrap, A. Santoro, L. Marris, C. J. Akerman, and G. Hinton · 2020
Cited alongside, same era.
Can the brain do backpropagation?—exact implementation of backpropagation in predictive coding networks
Y. Song, T. Lukasiewicz, Z. Xu, and R. Bogacz · 2020
Cited alongside, same era.
Hessian eigenspectra of more realistic nonlinear models
Z. Liao and M. W. Mahoney · 2021
Cited alongside, same era.
Predictive coding: a theoretical and experimental review
B. Millidge, A. Seth, and C. L. Buckley · 2021
Cited alongside, same era.
Later among the works it cites.
Width and depth limits commute in residual networks
S. Hayou and G. Yang · 2023
Later among the works it cites.
On the parameterization of second-order optimization effective towards the infinite width
S. Ishikawa and R. Karakida · 2023
Later among the works it cites.
Lecture notes on infinite-width limits of neural networks
C. Pehlevan and B. Bordelon · 2023
Later among the works it cites.
Brain-inspired computational intelligence via predictive coding
T. Salvatori, A. Mali, C. L. Buckley, T. Lukasiewicz, R. P. Rao, K. Friston, and A. Ororbia · 2023
Later among the works it cites.
Tensor programs ivb: Adaptive optimization in the infinite-width limit
G. Yang and E. Littwin · 2023
Later among the works it cites.
Tensor programs vi: Feature learning in infinite-depth neural networks
G. Yang, D. Yu, C. Zhu, and S. Hayou · 2023
Later among the works it cites.
Sparse maximal update parameterization: A holistic approach to sparse training dynamics
N. Dey, S. Bergsma, and J. Hestness · 2024
Later among the works it cites.
Effective sharpness aware minimization requires layerwise perturbation scaling
M. Haas, J. Xu, V. Cevher, and L. C. Vankadara · 2024
Later among the works it cites.
Commutative scaling of width and depth in deep neural networks
S. Hayou · 2024
Later among the works it cites.
Jpc: Flexible inference for predictive coding networks in jax
F. Innocenti, P. Kinghorn, W. Yun-Farmbrough, M. D. L. Varona, R. Singh, and C. L. Buckley · 2024
Later among the works it cites.
S. Ishikawa, R. Yokota, and R. Karakida · 2024
Later among the works it cites.
Benchmarking predictive coding networks–made simple
L. Pinchetti, C. Qi, O. Lokshyn, G. Olivers, C. Emde, M. Tang, A. M’Charrak, S. Frieder, B. Menzat, R. Bogacz, et al · 2024
Later among the works it cites.
Predictive coding networks and inference learning: Tutorial and survey
B. van Zwol, R. Jefferson, and E. L. Broek · 2024
Later among the works it cites.
Theoretical characterisation of the gauss-newton conditioning in neural networks
J. Zhao, S. P. Singh, and A. Lucchi · 2024
Later among the works it cites.
Don’t be lazy: Completep enables compute-efficient deep transformers
N. Dey, B. C. Zhang, L. Noci, M. Li, B. Bordelon, S. Bergsma, C. Pehlevan, B. Hanin, and J. Hestness · 2025
Closest in time.
Error optimization: Overcoming exponential signal decay in deep predictive coding networks
C. Goemaere, G. Oliviers, R. Bogacz, and T. Demeester · 2025
Closest in time.
Only strict saddles in the energy landscape of predictive coding networks?
F. Innocenti, E. M. Achour, R. Singh, and C. L. Buckley · 2025
Closest in time.
Super consistency of neural network landscapes and learning rate transfer
L. Noci, A. Meterez, T. Hofmann, and A. Orvieto · 2025
Closest in time.
Training deep predictive coding networks
C. Qi, T. Lukasiewicz, and T. Salvatori · 2025
Closest in time.