Towards Optimal One Pass Large Scale Learning with Averaged Stochastic Gradient Descent

On asymptotic normality in stochastic approximation

Václav Fabian · 1968

Earlier work this paper cites.

Asymptotically efficient stochastic approximation; the RM case

VÁclav Fabian · 1973

Earlier work this paper cites.

Acceleration of stochastic approximation by averaging

Boris T. Polyak and Anatoli. B. Juditsky · 1992

Earlier work this paper cites.

Gradient-based learning applied to document recognition

Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner · 1998

Earlier work this paper cites.

Adaptive method of realizing natural gradient learning for multilayer perceptrons

Shun-ichi Amari, Hyeyoung Park, and Kenji Fukumizu · 2000

Earlier work this paper cites.

RCV1: A new benchmark collection for text categorization research

David D. Lewis, Yiming Yang, Tony G. Rose, G., and Fan Li · 2004

Earlier work this paper cites.

Solving large scale linear prediction problems using stochastic gradient descent algorithms

Original

Tong Zhang · 2004

Earlier work this paper cites.

On-line learning for very large datasets

Léon Bottou and Yann LeCun · 2005

Cited alongside, same era.

Logarithmic regret algorithms for online convex optimization

Elad Hazan, Adam Kalai, and Satyen Kale Amit Agarwal · 2006

Cited alongside, same era.

Stochastic gradient descent on toy problems

Léon Bottou · 2007

Cited alongside, same era.

A stochastic quasi-newton method for online convex optimization

Nicol N. Schraudolph, Jin Yu, and Simon Günter · 2007

Cited alongside, same era.

Pegasos: Primal Estimated sub-GrAdient SOlver for SVM

Shai Shalev-Shwartz, Yoram Shinger, and Nathan Srebro · 2007

Cited alongside, same era.

The tradeoffs of large scale learning

Léon Bottou and Olivier Bousquet · 2008

Cited alongside, same era.

Topmoumoute online natural gradient algorithm

Nicolas Le Roux, Pierre-Antoine Manzagol, and Yoshua Bengio · 2008

Later among the works it cites.

Pascal large scale learning challenge

Soeren Sonnenburg, Vojtech Franc, Elad Yom-Tov, and Michele Sebag · 2008

Later among the works it cites.

SGD-QN: Careful quasi-Newton stochastic gradient descent

Antoine Bordes, Léon Bottou, and Patrick Gallinari · 2009

Later among the works it cites.

Sparse online learning via truncated gradient

John Langford, Lihong Li, and Tong Zhang · 2009

Later among the works it cites.

Robust stochastic approximation approach to stochastic programming

Arkadi Nemirovski, Anatoli Juditski, Guanghui Lan, and Alexander Shapiro · 2009

Later among the works it cites.

Stochastic methods for ℓ 1 \ell_{1} regularized loss minimization

Shai Shalev-Shwartz and Ambuj Tewari · 2009

Later among the works it cites.