Convergence guarantees for forward gradient descent in the linear regression model

Competitive learning: From interactive activation to adaptive resonance

Grossberg, S · 1987

Earlier work this paper cites.

The recent excitement about neural networks

Crick, F · 1989

Earlier work this paper cites.

Adaptive algorithms and stochastic approximations

Benveniste, A., Métivier, M., and Priouret, P · 1990

Earlier work this paper cites.

The Gauss-Markov theorem and random regressors

Shaffer, J. P · 1991

Earlier work this paper cites.

Acceleration of stochastic approximation by averaging

Polyak, B. T., and Juditsky, A. B · 1992

Earlier work this paper cites.

On the averaged stochastic approximation for linear regression

Györfi, L., and Walk, H · 1996

Earlier work this paper cites.

Stochastic approximation and recursive algorithms and applications

Kushner, H. J., and Yin, G. G · 2003

Earlier work this paper cites.

Introduction to stochastic search and optimization

Spall, J. C · 2003

Earlier work this paper cites.

On the central moments of the multidimensional Gaussian distribution

Triantafyllopoulos, K · 2003

Earlier work this paper cites.

Optimal rates of aggregation

Tsybakov, A. B · 2003

Earlier work this paper cites.

Introduction to derivative-free optimization

Conn, A. R., Scheinberg, K., and Vicente, L. N · 2009

Earlier work this paper cites.

Non-strongly-convex smooth stochastic approximation with convergence rate O (1/n)

Bach, F., and Moulines, E · 2013

Cited alongside, same era.

Random design analysis of ridge regression

Hsu, D., Kakade, S. M., and Zhang, T · 2014

Cited alongside, same era.

Optimal rates for zero-order convex optimization: the power of two function evaluations

Duchi, J. C., Jordan, M. I., Wainwright, M. J., and Wibisono, A · 2015

Cited alongside, same era.

Random synaptic feedback weights support error backpropagation for deep learning

Lillicrap, T. P., Cownden, D., Tweed, D. B., and Akerman, C. J · 2016

Cited alongside, same era.

Direct feedback alignment provides learning in deep neural networks

Nøkland, A · 2016

Cited alongside, same era.

Random gradient-free minimization of convex functions

Nesterov, Y., and Spokoiny, V · 2017

Backpropagation and the brain

Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J., and Hinton, G · 2020

Later among the works it cites.

A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications

Liu, S., Chen, P.-Y., Kailkhura, B., Zhang, G., Hero III, A. O., and Varshney, P. K · 2020

Later among the works it cites.

Gradients without backpropagation

Original

Baydin, A. G., Pearlmutter, B. A., Syme, D., Wood, F., and Torr, P · 2022

Later among the works it cites.

Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices

Mourtada, J · 2022

Later among the works it cites.

Scaling forward gradient with local losses

Original

Ren, M., Kornblith, S., Liao, R., and Hinton, G · 2022

Later among the works it cites.