Fetching the paper…
Reading the bibliography…
We present a novel method for convex unconstrained optimization that, without any modifications, ensures: (i) accelerated convergence rate for smooth objectives, (ii) standard convergence rate in the general (non-smooth) setting, and (iii) standard convergence rate in the stochastic optimization setting.
Problem complexity and method efficiency in optimization
A. Nemirovskii, D. B. Yudin, and E. Dawson · 1983
Earlier work this paper cites.
A method of solving a convex programming problem with convergence rate o (1/k2)
Y. Nesterov · 1983
Earlier work this paper cites.
Introductory lectures on convex optimization. 2004, 2003
Y. Nesterov · 2003
Earlier work this paper cites.
On the generalization ability of on-line learning algorithms
N. Cesa-Bianchi, A. Conconi, and C. Gentile · 2004
Earlier work this paper cites.
A fast iterative shrinkage-thresholding algorithm for linear inverse problems
A. Beck and M. Teboulle · 2009
Earlier work this paper cites.
Accelerated gradient methods for stochastic optimization and online learning
C. Hu, W. Pan, and J. T. Kwok · 2009
Earlier work this paper cites.
Adaptive bound optimization for online convex optimization
H. B. McMahan and M. Streeter · 2010
Earlier work this paper cites.
Dual averaging methods for regularized stochastic learning and online optimization
L. Xiao · 2010
Earlier work this paper cites.
A first-order primal-dual algorithm for convex problems with applications to imaging
A. Chambolle and T. Pock · 2011
Earlier work this paper cites.
Adaptive subgradient methods for online learning and stochastic optimization
J. Duchi, E. Hazan, and Y. Singer · 2011
Earlier work this paper cites.
An optimal method for stochastic composite optimization
G. Lan · 2012
Earlier work this paper cites.
A mathematical introduction to compressive sensing , volume 1
S. Foucart and H. Rauhut · 2013
Earlier work this paper cites.
On the importance of initialization and momentum in deep learning
I. Sutskever, J. Martens, G. Dahl, and G. Hinton · 2013
Cited alongside, same era.
Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
S. Shalev-Shwartz and T. Zhang · 2014
Cited alongside, same era.
A differential equation for modeling nesterov’s accelerated gradient method: Theory and insights
W. Su, S. Boyd, and E. Candes · 2014
Cited alongside, same era.
Fast inertial dynamics and fista algorithms in convex optimization. perturbation aspects
H. Attouch and Z. Chbani · 2015
Cited alongside, same era.
A geometric alternative to nesterov’s accelerated gradient descent
S. Bubeck, Y. T. Lee, and M. Singh · 2015
Cited alongside, same era.
On lower and upper bounds in smooth and strongly convex optimization
Y. Arjevani, S. Shalev-Shwartz, and O. Shamir · 2016
Later among the works it cites.
Analysis and design of optimization algorithms via integral quadratic constraints
L. Lessard, B. Recht, and A. Packard · 2016
Later among the works it cites.
Osga: a fast subgradient algorithm with optimal complexity
A. Neumaier · 2016
Later among the works it cites.
Regularized nonlinear acceleration
D. Scieur, A. d’Aspremont, and F. Bach · 2016
Later among the works it cites.
A variational perspective on accelerated methods in optimization
A. Wibisono, A. C. Wilson, and M. I. Jordan · 2016
Later among the works it cites.
Katyusha: The First Direct Acceleration of Stochastic Gradient Methods
Z. Allen-Zhu · 2017
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
From averaging to acceleration, there is only a step-size
N. Flammarion and F. Bach · 2015
Cited alongside, same era.
Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization
R. Frostig, R. Ge, S. Kakade, and A. Sidford · 2015
Cited alongside, same era.
Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization
G. Lan · 2015
Cited alongside, same era.
A universal catalyst for first-order optimization
H. Lin, J. Mairal, and Z. Harchaoui · 2015
Cited alongside, same era.
Universal gradient methods for convex optimization problems
Y. Nesterov · 2015
Cited alongside, same era.
Scale-free algorithms for online linear optimization
F. Orabona and D. Pál · 2015
Cited alongside, same era.
A universal primal-dual convex optimization framework
A. Yurtsever, Q. T. Dinh, and V. Cevher · 2015
Cited alongside, same era.
Later among the works it cites.
Linear Coupling: An Ultimate Unification of Gradient and Mirror Descent
Z. Allen-Zhu and L. Orecchia · 2017
Later among the works it cites.
Optimal rate of convergence of an ode associated to the fast gradient descent schemes for b> 0
J. Aujol and C. Dossal · 2017
Later among the works it cites.
Accelerated extra-gradient descent: A novel accelerated first-order method
J. Diakonikolas and L. Orecchia · 2017
Later among the works it cites.
Online to offline conversions, universality and adaptive minibatch sizes
K. Levy · 2017
Later among the works it cites.
On acceleration with noise-corrupted gradients
M. B. Cohen, J. Diakonikolas, and L. Orecchia · 2018
Closest in time.
Black-box reductions for parameter-free online learning in banach spaces
A. Cutkosky and F. Orabona · 2018
Closest in time.