Error Lower Bounds of Constant Step-size Stochastic Gradient Descent

Sgd: General analysis and improved rates

Original

Gower, R. M., Loizou, N., Qian, X., Sailanbayev, A., Shulgin, E., and Richtárik, P. (2019) · 1901

Earlier work this paper cites.

Solving large scale linear prediction problems using stochastic gradient descent algorithms

Zhang, T. (2004) · 2004

Earlier work this paper cites.

Large-scale machine learning with stochastic gradient descent

Bottou, L. (2010) · 2010

Earlier work this paper cites.

Parallelized stochastic gradient descent

Zinkevich, M., Weimer, M., Li, L., and Smola, A. J. (2010) · 2010

Earlier work this paper cites.

Non-asymptotic analysis of stochastic approximation algorithms for machine learning

Moulines, E. and Bach, F. R. (2011) · 2011

Cited alongside, same era.

Making gradient descent optimal for strongly convex stochastic optimization

Original

Rakhlin, A., Shamir, O., and Sridharan, K. (2011) · 2011

Cited alongside, same era.

Segnet: A deep convolutional encoder-decoder architecture for image segmentation

Badrinarayanan, V., Kendall, A., and Cipolla, R. (2017) · 2017

Cited alongside, same era.

Sgd and hogwild! convergence without the bounded gradients assumption

Original

Nguyen, L. M., Nguyen, P. H., van Dijk, M., Richtárik, P., Scheinberg, K., and Takáč, M. (2018a)

Cited in the paper.

Tight dimension independent lower bound on optimal expected convergence rate for diminishing step sizes in sgd

Original

Nguyen, P. H., Nguyen, L. M., and van Dijk, M. (2018b)

Cited in the paper.

Bridging the gap between constant step size stochastic gradient descent and markov chains

Original

Dieuleveut, A., Durmus, A., and Bach, F. (2017) · 2017

Later among the works it cites.

Optimization methods for large-scale machine learning

Bottou, L., Curtis, F. E., and Nocedal, J. (2018) · 2018

Later among the works it cites.

Lower error bounds for the stochastic gradient descent optimization algorithm: Sharp convergence rates for slowly and fast decaying learning rates

Jentzen, A. and Von Wurstemberger, P. (2019) · 2019

Closest in time.