Fetching the paper…
Reading the bibliography…
Forward gradients have been recently introduced to bypass backpropagation in autodifferentiation, while retaining unbiased estimators of true gradients.
“A Stochastic Approximation Method” Publisher: Institute of Mathematical Statistics
Herbert Robbins and Sutton Monro · 1951
Earlier work this paper cites.
“A simple automatic derivative evaluation program”
R.. Wengert · 1964
Earlier work this paper cites.
“A method for solving the convex programming problem with convergence rate O(1/k^2)”
Y. Nesterov · 1983
Earlier work this paper cites.
“A Simple Convergence Proof of Adam and Adagrad” arXiv: 2003.02395
Alexandre Défossez, Léon Bottou, Francis Bach and Nicolas Usunier · 2003
Earlier work this paper cites.
“Test Problems in Optimization” arXiv:1008.0549 [math]
Xin-She Yang · 2010
Earlier work this paper cites.
“Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning”
Eric Moulines and Francis Bach · 2011
Earlier work this paper cites.
“8 - Test Functions for Global Optimization: A Comprehensive Survey”
Momin Jamil, Xin-She Yang and Hans-Jürgen Zepernick · 2013
Cited alongside, same era.
“Autograd: Effortless Gradients in Numpy”, 2015, pp. 3
Dougal Maclaurin, David Duvenaud and Ryan Adams · 2015
Cited alongside, same era.
“Lecture notes: Statistical machine learning and convex optimization.”, 2016
Francis Bach · 2016
Cited alongside, same era.
“Best practices for comparing optimization algorithms” arXiv:1709.08242 [math]
Vahid Beiranvand, Warren Hare and Yves Lucet · 2017
Cited alongside, same era.
“Adam: A Method for Stochastic Optimization” arXiv:1412.6980 [cs]
Diederik. Kingma and Jimmy Ba · 2017
Cited alongside, same era.
“Automatic differentiation in machine learning: a survey” arXiv: 1502.05767
Atilim Baydin, Barak. Pearlmutter, Alexey Radul and Jeffrey Siskind · 2018
Later among the works it cites.
“Algorithms for Optimization”, 2019, pp. 520
Mykel Kochenderfer and Tim Wheeler · 2019
Later among the works it cites.
“A review of automatic differentiation and its efficient implementation” _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/widm.1305
Charles. Margossian · 2019
Later among the works it cites.
“AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients”
Juntang Zhuang et al · 2020
Later among the works it cites.
“Gradients without Backpropagation” arXiv: 2202.08587
Atılımüneş Baydin et al · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
“Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients” ISSN: 2640-3498
Lukas Balles and Philipp Hennig · 2018
Cited alongside, same era.
Closest in time.
“Optimization & Eye Pleasure: 78 Benchmark Test Functions for Single Objective Optimization”
Axel Thevenot · 2022
Closest in time.