Fetching the paper…

Reading the bibliography…

2018

An improvement of the convergence proof of the ADAM-Optimizer

Bock, Sebastian, Goppold, Josef, Weiß, Martin

Original
PDF
Post on X

Understand

A common way to train neural networks is the Backpropagation.

This algorithm includes a gradient descent method, which needs an adaptive step size.
In the area of neural networks, the ADAM-Optimizer is one of the most popular adaptive step size methods.
It was invented in \cite{Kingma.2015} by Kingma and Ba.

An improvement of the convergence proof of the ADAM-Optimizer · Around

Built on

D. E. Rumelhart and J. L. McClelland, “Learning Internal Representations by Error Propagation,” Parallel Distributed Processing:Explorations in the Microstructure of Cognition: Foundations , pp. 318–362, 1987. [Online]. Available: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6302929
Original
1987
Earlier work this paper cites.
R. Kruse, Computational Intelligence: Eine methodische Einführung in Künstliche Neuronale Netze, Evolutionäre Algorithmen, Fuzzy-Systeme und Bayes-Netze , 1st ed., ser. Computational Intelligence. Wiesbaden: Vieweg + Teubner, 2011
2011
Earlier work this paper cites.

Similar

D. P. Kingma and J. L. Ba, Adam: A Method for stochastic Optimization . San Diego: The International Conference on Learning Representations (ICLR), 2015
2015
Cited alongside, same era.
O. Forster, Analysis , 12th ed., ser. Grundkurs Mathematik. Braunschweig and Wiesbaden: Vieweg, 2016, vol. 1
2016
Cited alongside, same era.
S. Ruder, “An overview of gradient descent optimization algorithms,” cite arxiv:1609.04747Comment: 12 pages, 6 figures. [Online]. Available: http://arxiv.org/abs/1609.04747
Original
Cited in the paper.

Then

J. Goppold, “Identifikation von Serverfehlern mittels Support Vector Machines und künstlichen neuronalen Netzen,” Regensburg, 2017
2017
Later among the works it cites.
S. Bock, “Rotationsermittlung von Bauteilen basierend auf neuronalen Netzen,” Regensburg, 2017
2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…