Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias

Phd thesis: Modeles connexionnistes de l’apprentissage (connectionist learning models)

Y. Lecun · 1987

Earlier work this paper cites.

Backpropagation without weight transport

J. F. Kolen and J. B. Pollack · 1994

Earlier work this paper cites.

The graph neural network model

F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini · 2008

Earlier work this paper cites.

Learning multiple layers of features from tiny images

A. Krizhevsky, G. Hinton, et al · 2009

Earlier work this paper cites.

How auto-encoders could provide credit assignment in deep networks via target propagation

Original

Y. Bengio · 2014

Earlier work this paper cites.

Dropout: a simple way to prevent neural networks from overfitting

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov · 2014

Earlier work this paper cites.

Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

K. He, X. Zhang, S. Ren, and J. Sun · 2015

Earlier work this paper cites.

Random synaptic feedback weights support error backpropagation for deep learning

T. P. Lillicrap, D. Cownden, D. B. Tweed, and C. J. Akerman · 2016

Earlier work this paper cites.

Sgdr: Stochastic gradient descent with warm restarts

Original

I. Loshchilov and F. Hutter · 2016

Earlier work this paper cites.

Automatic differentiation in pytorch

A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer · 2017

Cited alongside, same era.

Equilibrium propagation: Bridging the gap between energy-based models and backpropagation

B. Scellier and Y. Bengio · 2017

Cited alongside, same era.

Equivalent-accuracy accelerated neural-network training using analogue memory

S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. di Nolfo, S. Sidler, M. Giordano, M. Bodini, N. C. Farinha, et al · 2018

Cited alongside, same era.

Assessing the scalability of biologically-motivated deep learning algorithms and architectures

S. Bartunov, A. Santoro, B. Richards, L. Marris, G. E. Hinton, and T. Lillicrap · 2018

Cited alongside, same era.

In-memory and error-immune differential rram implementation of binarized deep neural networks

M. Bocquet, T. Hirztlin, J.-O. Klein, E. Nowak, E. Vianello, J.-M. Portal, and D. Querlioz · 2018

Cited alongside, same era.

Deep learning without weight transport

M. Akrout, C. Wilson, P. Humphreys, T. Lillicrap, and D. B. Tweed · 2019

Later among the works it cites.

Updates of equilibrium prop match gradients of backprop through time in an rnn with static input

Original

M. Ernoult, J. Grollier, D. Querlioz, Y. Bengio, and B. Scellier · 2019

Later among the works it cites.

Training a spiking neural network with equilibrium propagation

P. O’Connor, E. Gavves, and M. Welling · 2019

Later among the works it cites.

A deep learning framework for neuroscience

B. A. Richards, T. P. Lillicrap, P. Beaudoin, Y. Bengio, R. Bogacz, A. Christensen, C. Clopath, R. P. Costa, A. de Berker, S. Ganguli, et al · 2019

Later among the works it cites.

Equivalence of equilibrium propagation and recurrent backpropagation

B. Scellier and Y. Bengio · 2019

Later among the works it cites.

Equilibrium propagation with continual weight updates