Fetching the paper…
Reading the bibliography…
In this paper, we provide an overview of first-order and second-order variants of the gradient descent method that are commonly used in machine learning.
Information theory and statistics
Solomon Kullback · 1968
Earlier work this paper cites.
Statistical decision rules and optimal inference , volume 53 of Translations of Mathematical Monographs
N. N. Čencov · 1982
Earlier work this paper cites.
Neural learning in structured parameter spaces-natural riemannian gradient
Shun-ichi Amari · 1997
Earlier work this paper cites.
Natural gradient works efficiently in learning
Shun-ichi Amari · 1998
Earlier work this paper cites.
The tradeoffs of large scale learning
Léon Bottou and Olivier Bousquet · 2008
Earlier work this paper cites.
Natural actor-critic
Jan Peters and Stefan Schaal · 2008
Earlier work this paper cites.
Training deep and recurrent networks with hessian-free optimization
James Martens and Ilya Sutskever · 2012
Cited alongside, same era.
Objective improvement in information-geometric optimization
Youhei Akimoto and Yann Ollivier · 2013
Cited alongside, same era.
Revisiting natural gradient for deep networks
Razvan Pascanu and Yoshua Bengio · 2013
Cited alongside, same era.
New insights and perspectives on the natural gradient method
James Martens · 2014
Cited alongside, same era.
Riemannian metrics for neural networks I: Feedforward networks
Yann Ollivier · 2015
Cited alongside, same era.
Trust region policy optimization
John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, and Pieter Abbeel · 2015
Later among the works it cites.
Scalable trust-region method for deep reinforcement learning using kronecker-factored approximation
Yuhuai Wu, Elman Mansimov, Roger B Grosse, Shun Liao, and Jimmy Ba · 2017
Later among the works it cites.
Optimization methods for large-scale machine learning
Léon Bottou, Frank E Curtis, and Jorge Nocedal · 2018
Closest in time.
Fast approximate natural gradient descent in a kronecker-factored eigenbasis
Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, and Pascal Vincent · 2018
Closest in time.
Policy search in continuous action domains: an overview
Olivier Sigaud and Freek Stulp · 2018
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Closest in time.