Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes

Stochastic Approximation and Recursive Algorithms and Applications

Kushner, H. and Yin, G · 2003

Earlier work this paper cites.

Online convex programming and generalized infinitesimal gradient ascent

Zinkevich, M · 2003

Earlier work this paper cites.

Solving large scale linear prediction problems using stochastic gradient descent algorithms

Zhang, T · 2004

Earlier work this paper cites.

Logarithmic regret algorithms for online convex optimization

Hazan, E., Agarwal, A., and Kale, S · 2007

Earlier work this paper cites.

Stochastic convex optimization

Shalev-Shwartz, S., Shamir, O., Srebro, N., and Sridharan, K · 2009

Earlier work this paper cites.

Non-asymptotic analysis of stochastic approximation algorithms for machine learning

Bach, F. and Moulines, E · 2011

Cited alongside, same era.

Beyond the regret minimization barrier: An optimal algorithm for stochastic strongly-convex optimization

Hazan, E. and Kale, S · 2011

Cited alongside, same era.

Making gradient descent optimal for strongly convex stochastic optimization

Original

Rakhlin, A., Shamir, O., and Sridharan, K · 2011

Cited alongside, same era.

Pegasos: primal estimated sub-gradient solver for svm

Shalev-Shwartz, S., Singer, Y., Srebro, N., and Cotter, A · 2011

Cited alongside, same era.

Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization

Agarwal, A., Bartlett, P., Ravikumar, P., and Wainwright, M · 2012

Closest in time.

A simpler approach to obtaining an o(1/t) convergence rate for projected stochastic subgradient descent

Original

Lacoste-Julien, S., Schmidt, M., and Bach, F · 2012

Closest in time.

Stochastic smoothing for nonsmooth minimizations: Accelerating sgd by exploiting structure

Ouyang, H. and Gray, A · 2012

Closest in time.

Is averaging needed for strongly convex stochastic gradient descent?

Shamir, O · 2012

Closest in time.