Fetching the paper…
Reading the bibliography…
In this short note, I show how to adapt to H\"{o}lder smoothness using normalized gradients in a black-box way.
Adaptive gradient descent without descent
K. Mishchenko and Y. Malitsky · 1910
Earlier work this paper cites.
A modern introduction to online learning
F. Orabona · 1912
Earlier work this paper cites.
A method for unconstrained convex minimization problem with the rate of convergence O ( 1 / k 2 ) O(1/k^{2})
Y. Nesterov · 1983
Earlier work this paper cites.
Approximate solutions to Markov decision processes
G. J. Gordon · 1999
Earlier work this paper cites.
Introductory lectures on convex optimization: A basic course , volume 87
Y. Nesterov · 2004
Earlier work this paper cites.
Online learning meets optimization in the dual
S. Shalev-Shwartz and Y. Singer · 2006
Earlier work this paper cites.
Competing in the dark: An efficient algorithm for bandit linear optimization
J. D. Abernethy, E. Hazan, and A. Rakhlin · 2008
Earlier work this paper cites.
Extracting certainty from uncertainty: Regret bounded by variation in costs
E. Hazan and S. Kale · 2008
Earlier work this paper cites.
Primal-dual subgradient methods for convex problems
Y. Nesterov · 2009
Earlier work this paper cites.
Less regret via online conditioning
M. Streeter and H. B. McMahan · 2010
Earlier work this paper cites.
No-regret algorithms for unconstrained online convex optimization
M. Streeter and B. McMahan · 2012
Earlier work this paper cites.
Unconstrained online linear learning in Hilbert spaces: Minimax algorithms and normal approximations
H. B. McMahan and F. Orabona · 2014
Earlier work this paper cites.
Simultaneous model selection and optimization through parameter-free stochastic learning
F. Orabona · 2014
Cited alongside, same era.
Universal gradient methods for convex optimization problems
Y. Nesterov · 2015
Cited alongside, same era.
Scale-free algorithms for online linear optimization
F. Orabona and D. Pál · 2015
Cited alongside, same era.
Coin betting and parameter-free online learning
F. Orabona and D. Pál · 2016
Cited alongside, same era.
Dual averaging methods for regularized stochastic learning and online optimization
L. Xiao · 2016
Cited alongside, same era.
On the convergence of stochastic gradient descent with adaptive stepsizes
X. Li and F. Orabona · 2019
Later among the works it cites.
Lipschitz and comparator-norm adaptivity in online learning
Z. Mhammedi and W. M Koolen · 2020
Later among the works it cites.
Parameter-free stochastic optimization of variationally coherent functions
F. Orabona and D. Pál · 2021
Later among the works it cites.
Making SGD parameter-free
Y. Carmon and O. Hinder · 2022
Later among the works it cites.
Implicit parameter-free online learning with truncated linear models
K. Chen, A. Cutkosky, and F. Orabona · 2022
Later among the works it cites.
On optimal universal first-order methods for minimizing heterogeneous sums
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Online learning without prior information
A. Cutkosky and K. Boahen · 2017
Cited alongside, same era.
Parameter-free online learning via model selection
D. J. Foster, S. Kale, M. Mohri, and K. Sridharan · 2017
Cited alongside, same era.
Online to offline conversions, universality and adaptive minibatch sizes
K. Levy · 2017
Cited alongside, same era.
Training deep networks without learning rates through coin betting
F. Orabona and T. Tommasi · 2017
Cited alongside, same era.
Black-box reductions for parameter-free online learning in Banach spaces
A. Cutkosky and F. Orabona · 2018
Cited alongside, same era.
Convergence rates for deterministic and stochastic subgradient methods without Lipschitz continuity
B. Grimmer · 2019
Cited alongside, same era.
Parameter-free online convex optimization with sub-exponential noise
K.-S. Jun and F. Orabona · 2019
Cited alongside, same era.
B. Grimmer · 2022
Later among the works it cites.
Parameter-free regret in high probability with heavy tails
J. Zhang and A. Cutkosky · 2022
Later among the works it cites.
PDE-based optimal strategy for unconstrained online learning
Z. Zhang, A. Cutkosky, and I. Paschalidis · 2022
Later among the works it cites.
Parameter-free projected gradient descent
E. Chzhen, C. Giraud, and G. Stoltz · 2023
Closest in time.
DoG is SGD’s best friend: A parameter-free dynamic step size schedule
M. Ivgi, O. Hinder, and Y. Carmon · 2023
Closest in time.
Unconstrained online learning with unbounded losses
A. Jacobsen and A. Cutkosky · 2023
Closest in time.
DoWG unleashed: An efficient universal parameter-free gradient descent method
A. Khaled, K. Mishchenko, and C. Jin · 2023
Closest in time.