Fetching the paper…
Reading the bibliography…
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.
Variational analysis , volume 317
R. T. Rockafellar and R. J.-B. Wets · 2009
Earlier work this paper cites.
Adaptive subgradient methods for online learning and stochastic optimization
J. Duchi, E. Hazan, and Y. Singer · 2011
Earlier work this paper cites.
Lecture 6.5-RMSprop: Divide the gradient by a running average of its recent magnitude
T. Tieleman and G. Hinton · 2012
Earlier work this paper cites.
Stochastic first-and zeroth-order methods for nonconvex stochastic programming
S. Ghadimi and G. Lan · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
D. P. Kingma and J. Ba · 2015
Earlier work this paper cites.
Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
S. Ghadimi, G. Lan, and H. Zhang · 2016
Earlier work this paper cites.
Online to offline conversions, universality and adaptive minibatch sizes
K. Levy · 2017
Earlier work this paper cites.
Imposing hard constraints on deep networks: Promises and limitations
P. Márquez-Neila, M. Salzmann, and P. Fua · 2017
Earlier work this paper cites.
Error bounds, quadratic growth, and linear convergence of proximal methods
D. Drusvyatskiy and A. S. Lewis · 2018
Earlier work this paper cites.
Stochastic methods for composite and weakly convex optimization problems
J. C. Duchi and F. Ruan · 2018
Earlier work this paper cites.
Black-box adversarial attacks with limited queries and information
A. Ilyas, L. Engstrom, A. Athalye, and J. Lin · 2018
Cited alongside, same era.
Online adaptive methods, universality and acceleration
K. Y. Levy, A. Yurtsever, and V. Cevher · 2018
Cited alongside, same era.
Towards deep learning models resistant to adversarial attacks
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu · 2018
Cited alongside, same era.
On the convergence of Adam and beyond
S. J. Reddi, S. Kale, and S. Kumar · 2018
Cited alongside, same era.
Weighted adagrad with unified momentum
F. Zou, L. Shen, Z. Jie, J. Sun, and W. Liu · 2018
Cited alongside, same era.
Convergence analysis of a momentum algorithm with adaptive step size for non convex optimization
On connections between constrained optimization and reinforcement learning
N. Vieillard, O. Pietquin, and M. Geist · 2019
Later among the works it cites.
Adagrad stepsizes: Sharp convergence over nonconvex landscapes
R. Ward, X. Wu, and L. Bottou · 2019
Later among the works it cites.
A sufficient condition for convergences of Adam and RMSprop
F. Zou, L. Shen, Z. Jie, W. Zhang, and W. Liu · 2019
Later among the works it cites.
A new regret analysis for Adam-type algorithms
A. Alacaoglu, Y. Malitsky, P. Mertikopoulos, and V. Cevher · 2020
Closest in time.
Closing the generalization gap of adaptive gradient methods in training deep neural networks
J. Chen, D. Zhou, Y. Tang, Z. Yang, Y. Cao, and Q. Gu · 2020
Closest in time.
Stochastic subgradient method converges on tame functions
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
A. Barakat and P. Bianchi · 2019
Cited alongside, same era.
Stochastic model-based minimization of weakly convex functions
D. Davis and D. Drusvyatskiy · 2019
Cited alongside, same era.
Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems
D. Davis and B. Grimmer · 2019
Cited alongside, same era.
Efficiency of minimizing compositions of convex functions and smooth maps
D. Drusvyatskiy and C. Paquette · 2019
Cited alongside, same era.
On the convergence of stochastic gradient descent with adaptive stepsizes
X. Li and F. Orabona · 2019
Cited alongside, same era.
On the convergence of a class of adam-type algorithms for non-convex optimization
X. Chen, S. Liu, R. Sun, and M. Hong
Cited in the paper.
ZO-AdaMM: Zeroth-order adaptive momentum method for black-box optimization
X. Chen, S. Liu, K. Xu, X. Li, X. Lin, M. Hong, and D. Cox
Cited in the paper.
D. Davis, D. Drusvyatskiy, S. Kakade, and J. D. Lee · 2020
Closest in time.
On the convergence of Adam and Adagrad
A. Défossez, L. Bottou, F. Bach, and N. Usunier · 2020
Closest in time.
Convergence of a stochastic gradient method with momentum for nonsmooth nonconvex optimization
V. V. Mai and M. Johansson · 2020
Closest in time.
On complexity of finding stationary points of nonsmooth nonconvex functions
J. Zhang, H. Lin, S. Sra, and A. Jadbabaie · 2020
Closest in time.