Fetching the paper…
Reading the bibliography…
We present new algorithms for online convex optimization over unbounded domains that obtain parameter-free regret in high-probability given access only to potentially heavy-tailed subgradient estimates.
Etude critique de la notion de collectif, gauthier-villars, paris, 1939
J. Ville · 1939
Earlier work this paper cites.
Online convex programming and generalized infinitesimal gradient ascent
M. Zinkevich · 2003
Earlier work this paper cites.
Prediction, learning, and games
N. Cesa-Bianchi and G. Lugosi · 2006
Earlier work this paper cites.
Hoeffding’s inequality in game-theoretic probability
V. Vovk · 2007
Earlier work this paper cites.
Online learning and online convex optimization
S. Shalev-Shwartz · 2011
Earlier work this paper cites.
Freedman’s inequality for matrix martingales
J. Tropp · 2011
Earlier work this paper cites.
Bandits with heavy tail
S. Bubeck, N. Cesa-Bianchi, and G. Lugosi · 2013
Earlier work this paper cites.
Sharp finite-time iterated-logarithm martingale concentration
A. Balsubramani · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
D. Kingma and J. Ba · 2014
Earlier work this paper cites.
Coin betting and parameter-free online learning
F. Orabona and D. Pál · 2016
Earlier work this paper cites.
Online learning without prior information
A. Cutkosky and K. Boahen · 2017
Earlier work this paper cites.
Parameter-free online learning via model selection
D. J. Foster, S. Kale, M. Mohri, and K. Sridharan · 2017
Earlier work this paper cites.
Algorithms and Lower Bounds for Parameter-free Online Learning
A. Cutkosky · 2018
Earlier work this paper cites.
Black-box reductions for parameter-free online learning in banach spaces
A. Cutkosky and F. Orabona · 2018
Cited alongside, same era.
On the convergence of adam and beyond
S. J. Reddi, S. Kale, and S. Kumar · 2018
Cited alongside, same era.
Online control with adversarial disturbances
N. Agarwal, B. Bullins, E. Hazan, S. Kakade, and K. Singh · 2019
Cited alongside, same era.
Combining online learning guarantees
A. Cutkosky · 2019
Cited alongside, same era.
Simple and optimal high-probability bounds for strongly-convex stochastic gradient descent
N. J. Harvey, C. Liaw, and S. Randhawa · 2019
Cited alongside, same era.
L. Madden, E. Dall’Anese, and S. Becker · 2020
Later among the works it cites.
Lipschitz and comparator-norm adaptivity in online learning
Z. Mhammedi and W. M. Koolen · 2020
Later among the works it cites.
Estimating means of bounded random variables by betting
I. Waudby-Smith and A. Ramdas · 2020
Later among the works it cites.
Why are adaptive methods good for attention models?
J. Zhang, S. P. Karimireddy, A. Veit, S. Kim, S. Reddi, S. Kumar, and S. Sra · 2020
Later among the works it cites.
Towards theoretically understanding why sgd generalizes better than adam in deep learning
P. Zhou, J. Feng, C. Ma, C. Xiong, S. C. H. Hoi, et al · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
E. Hazan · 2019
Cited alongside, same era.
Parameter-free online convex optimization with sub-exponential noise
K.-S. Jun and F. Orabona · 2019
Cited alongside, same era.
Adaptive scale-invariant online algorithms for learning linear models
M. Kempka, W. Kotlowski, and M. K. Warmuth · 2019
Cited alongside, same era.
A modern introduction to online learning
F. Orabona · 2019
Cited alongside, same era.
User-specified local differential privacy in unconstrained adaptive online learning
D. van der Hoeven · 2019
Cited alongside, same era.
Stochastic optimization with heavy-tailed noise via accelerated gradient clipping
E. Gorbunov, M. Danilova, and A. Gasnikov · 2020
Cited alongside, same era.
A high probability analysis of adaptive sgd with momentum
X. Li and F. Orabona · 2020
Cited alongside, same era.
Impossible tuning made possible: A new expert algorithm and its applications
L. Chen, H. Luo, and C.-Y. Wei · 2021
Later among the works it cites.
High-probability bounds for non-convex stochastic optimization with heavy tails
A. Cutkosky and H. Mehta · 2021
Later among the works it cites.
Time-uniform, nonparametric, nonasymptotic confidence sequences
S. R. Howard, A. Ramdas, J. McAuliffe, and J. Sekhon · 2021
Later among the works it cites.
Tight concentrations and confidence sequences from the regret of universal portfolio
F. Orabona and K.-S. Jun · 2021
Later among the works it cites.
Making sgd parameter-free
Y. Carmon and O. Hinder · 2022
Closest in time.
High probability bounds for a class of nonconvex algorithms with adagrad stepsize
A. Kavis, K. Y. Levy, and V. Cevher · 2022
Closest in time.
Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance
N. M. Vural, L. Yu, K. Balasubramanian, S. Volgushev, and M. A. Erdogdu · 2022
Closest in time.