Online Regularized Nonlinear Acceleration

Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods

Gene H. Golub and Richard S. Varga · 1961

Earlier work this paper cites.

Iterative procedures for nonlinear integral equations

Donald G. Anderson · 1965

Earlier work this paper cites.

A polynomial extrapolation method for finding limits and antilimits of vector sequences

Stan Cabay and LW Jackson · 1976

Earlier work this paper cites.

A method of solving a convex programming problem with convergence rate O ⁡ ( 1 / k 2 ) {O}(1/k^{2})

Yurii Nesterov · 1983

Earlier work this paper cites.

Analysis of hidden units in a layered network trained to classify sonar targets

R. Paul Gorman and Terrence J. Sejnowski · 1988

Earlier work this paper cites.

Cubic regularization of Newton method and its global performance

Yurii Nesterov and Boris T Polyak · 2006

Earlier work this paper cites.

Nonlinear Equations

Jorge Nocedal and Stephen J. Wright · 2006

Earlier work this paper cites.

Sido: A phamacology dataset

I. Guyon · 2008

Cited alongside, same era.

Feature extraction: foundations and applications , volume 207

Isabelle Guyon, Steve Gunn, Masoud Nikravesh, and Lofti A Zadeh · 2008

Cited alongside, same era.

Nonsmooth optimization via bfgs

Adrian S. Lewis and Michael L. Overton · 2009

Cited alongside, same era.

Convolutional networks and applications in vision

Yann LeCun, Koray Kavukcuoglu, and Clément Farabet · 2010

Cited alongside, same era.

Matrix Computations , volume 3

Gene H. Golub and Charles F. Van Loan · 2012

Cited alongside, same era.

Introductory lectures on convex optimization: A basic course , volume 87

Yurii Nesterov · 2013

Cited alongside, same era.

Nonlinear acceleration of stochastic algorithms

SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives

Aaron Defazio, Francis Bach, and Simon Lacoste-Julien · 2014

Later among the works it cites.

A stochastic quasi-newton method for large-scale optimization

Richard H. Byrd, Samantha L. Hansen, Jorge Nocedal, and Yoram Singer · 2016

Later among the works it cites.

Restarting accelerated gradient methods with a rough strong convexity estimate

Original

Olivier Fercoq and Zheng Qu · 2016

Later among the works it cites.

Regularized nonlinear acceleration

Damien Scieur, Alexandre d’Aspremont, and Francis Bach · 2016

Later among the works it cites.

Katyusha: The first direct acceleration of stochastic gradient methods

Zeyuan Allen-Zhu · 2017

Later among the works it cites.

Nonlinear acceleration of primal-dual algorithms