Fetching the paper…
Reading the bibliography…
Evolutionary Strategies (ES) are known to be an effective black-box optimization technique for deep neural networks when the true gradients cannot be computed, such as in Reinforcement Learning.
Evolution strategy: Optimization of technical systems by means of biological evolution
Ingo Rechenberg · 1973
Earlier work this paper cites.
Evolutionsstrategien für die numerische optimierung
Hans-Paul Schwefel · 1977
Earlier work this paper cites.
Learning internal representations by error propagation
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams · 1985
Earlier work this paper cites.
Q-learning
Christopher JCH Watkins and Peter Dayan · 1992
Earlier work this paper cites.
Estimating or propagating gradients through stochastic neurons for conditional computation
Yoshua Bengio, Nicholas Léonard, and Aaron Courville · 2013
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton · 2013
Earlier work this paper cites.
Random feedback weights support learning in deep neural networks
Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman · 2014
Earlier work this paper cites.
Natural evolution strategies
Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber · 2014
Cited alongside, same era.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2014
Cited alongside, same era.
Incorporating nesterov momentum into adam
Timothy Dozat · 2016
Cited alongside, same era.
An overview of gradient descent optimization algorithms
Sebastian Ruder · 2016
Cited alongside, same era.
The cma evolution strategy: A tutorial
Nikolaus Hansen · 2016
Cited alongside, same era.
Random gradient-free minimization of convex functions
Structured evolution with compact architectures for scalable policy optimization
Krzysztof Choromanski, Mark Rowland, Vikas Sindhwani, Richard E Turner, and Adrian Weller · 2018
Later among the works it cites.
Evolutionary stochastic gradient descent for optimization of deep neural networks
Xiaodong Cui, Wei Zhang, Zoltán Tüske, and Michael Picheny · 2018
Later among the works it cites.
Evolved policy gradients
Rein Houthooft, Yuhua Chen, Phillip Isola, Bradly Stadie, Filip Wolski, OpenAI Jonathan Ho, and Pieter Abbeel · 2018
Later among the works it cites.
Recurrent world models facilitate policy evolution
David Ha and Jürgen Schmidhuber · 2018
Later among the works it cites.
Learned optimizers that outperform on wall-clock and validation loss
Luke Metz, Niru Maheswaranathan, Jeremy Nixon, Daniel Freeman, and Jascha Sohl-dickstein · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yurii Nesterov and Vladimir Spokoiny · 2017
Cited alongside, same era.
Evolution strategies as a scalable alternative to reinforcement learning
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever · 2017
Cited alongside, same era.
Johannes Lengler and Angelika Steger · 2018
Later among the works it cites.
Guided evolutionary strategies: augmenting random search with surrogate gradients
Niru Maheswaranathan, Luke Metz, George Tucker, Dami Choi, and Jascha Sohl-Dickstein · 2019
Closest in time.