Fetching the paper…
Reading the bibliography…
We show that a particular form of target propagation, i.e., relying on learned inverses of each layer, which is differential, i.e., where the target is a small perturbation of the forward propagation, gives rise to an update rule which corresponds to an approximate Gauss-Newton gradient-based optimization, without requiring the manipulation or inversion of large matrices.
Learning processes in an asymmetric threshold network
Yann LeCun · 1986
Earlier work this paper cites.
Competitive learning: From interactive activation to adaptive resonance
Stephen Grossberg · 1987
Earlier work this paper cites.
Phd thesis: Modeles connexionnistes de l’apprentissage (connectionist learning models)
Yann Lecun · 1987
Earlier work this paper cites.
A theoretical framework for back-propagation
Yann LeCun, D Touresky, G Hinton, and T Sejnowski · 1988
Earlier work this paper cites.
The recent excitement about neural networks
Francis Crick · 1989
Earlier work this paper cites.
A cost function for internal representations
Anders Krogh, CI Thorbergsson, and John A Hertz · 1990
Earlier work this paper cites.
The" wake-sleep" algorithm for unsupervised neural networks
Geoffrey E Hinton, Peter Dayan, Brendan J Frey, and Radford M Neal · 1995
Earlier work this paper cites.
The role of acetylcholine in learning and memory
Michael E Hasselmo · 2006
Earlier work this paper cites.
How auto-encoders could provide credit assignment in deep networks via target propagation
Yoshua Bengio · 2014
Cited alongside, same era.
Distributed optimization of deeply nested systems
Miguel Carreira-Perpinan and Weiran Wang · 2014
Cited alongside, same era.
How important is weight symmetry in backpropagation?
Qianli Liao, Joel Z Leibo, and Tomaso Poggio · 2015
Cited alongside, same era.
Random synaptic feedback weights support error backpropagation for deep learning
Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman · 2016
Cited alongside, same era.
Training neural networks without gradients: A scalable admm approach
Gavin Taylor, Ryan Burmeister, Zheng Xu, Bharat Singh, Ankit Patel, and Tom Goldstein · 2016
Cited alongside, same era.
Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding
N. R. Ke, A. Goyal, O. Bilaniuk, J. Binas, M. C. Mozer, C. Pal, and Y. Bengio · 2018
Later among the works it cites.
Beyond backprop: Online alternating minimization with auxiliary variables
Anna Choromanska, Benjamin Cowen, Sadhana Kumaravel, Ronny Luss, Mattia Rigotti, Irina Rish, Paolo Diachille, Viatcheslav Gurev, Brian Kingsbury, Ravi Tejwani, et al · 2019
Later among the works it cites.
Biologically motivated algorithms for propagating local target representations
Alexander G Ororbia and Ankur Mali · 2019
Later among the works it cites.
Gait-prop: A biologically plausible learning rule derived from backpropagation of error
Nasir Ahmad, Marcel AJ van Gerven, and Luca Ambrogioni · 2020
Closest in time.
Untangling tradeoffs between recurrence and self-attention in neural networks
Giancarlo Kerg, Bhargav Kanuparthi, Anirudh Goyal, Kyle Goyette, Yoshua Bengio, and Guillaume Lajoie · 2020
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yoshua Bengio · 2017
Cited alongside, same era.
Equilibrium propagation: Bridging the gap between energy-based models and backpropagation
Benjamin Scellier and Yoshua Bengio · 2017
Cited alongside, same era.
Assessing the scalability of biologically-motivated deep learning algorithms and architectures
Sergey Bartunov, Adam Santoro, Blake A. Richards, Geoffrey E. Hinton, and Timothy Lillicrap · 2018
Cited alongside, same era.
Deeply-supervised nets
Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu
Cited in the paper.
Difference target propagation
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio
Cited in the paper.
Closest in time.
Backpropagation and the brain
Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hinton · 2020
Closest in time.
Biological credit assignment through dynamic inversion of feedforward networks
William F Podlaski and Christian K Machens · 2020
Closest in time.