Stochastic gradient methods (SGMs) have been extensively used for solving stochastic problems or large-scale machine learning problems.
Reading the bibliography…
A stochastic approximation method
H. Robbins and S. Monro · 1951
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky · 2009
Earlier work this paper cites.
Online learning for matrix factorization and sparse coding
J. Mairal, F. Bach, J. Ponce, and G. Sapiro · 2010
Earlier work this paper cites.
LIBSVM: a library for support vector machines
C.-C. Chang and C.-J. Lin · 2011
Earlier work this paper cites.
Stochastic first and zeroth-order methods for nonconvex stochastic programming
S. Ghadimi and G. Lan · 2013
Earlier work this paper cites.
Memory limited, streaming PCA
I. Mitliagkas, C. Caramanis, and P. Jain · 2013
Earlier work this paper cites.
Sparse bilinear logistic regression
J. V. Shi, Y. Xu, and R. G. Baraniuk · 2014
Earlier work this paper cites.
Striving for simplicity: The all convolutional net
J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller · 2014
Earlier work this paper cites.
Sparse convolutional neural networks
B. Liu, M. Wang, H. Foroosh, M. Tappen, and M. Pensky · 2015
Earlier work this paper cites.
Block stochastic gradient iteration for convex and nonconvex optimization
Y. Xu and W. Yin · 2015
Earlier work this paper cites.
Variance reduction for faster non-convex optimization
Z. Allen-Zhu and E. Hazan · 2016
Earlier work this paper cites.
Accelerated gradient methods for nonconvex nonlinear and stochastic programming
S. Ghadimi and G. Lan · 2016
Earlier work this paper cites.
Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
S. Ghadimi, G. Lan, and H. Zhang · 2016
Earlier work this paper cites.
Asynchronous stochastic gradient descent with variance reduction for non-convex optimization
Z. Huo and H. Huang · 2016
Cited alongside, same era.
On large-batch training for deep learning: Generalization gap and sharp minima
N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang · 2016
Cited alongside, same era.
Stochastic variance reduction for nonconvex optimization
S. J. Reddi, A. Hefny, S. Sra, B. Póczos, and A. Smola · 2016
Cited alongside, same era.
Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization
S. J. Reddi, S. Sra, B. Poczos, and A. J. Smola · 2016
Cited alongside, same era.
Online nonnegative matrix factorization with outliers
R. Zhao and V. Y. Tan · 2016
Cited alongside, same era.
Non-convex finite-sum optimization via scsg methods
L. Lei, C. Ju, J. Chen, and M. I. Jordan · 2017
Cited alongside, same era.
Sarah: A novel method for machine learning problems using stochastic recursive gradient
L. M. Nguyen, J. Liu, K. Scheinberg, and M. Takáč · 2017
Cited alongside, same era.
Group sparse regularization for deep neural networks
S. Scardapane, D. Comminiello, A. Hussain, and A. Uncini · 2017
Cited alongside, same era.
Natasha 2: Faster non-convex optimization than sgd
Z. Allen-Zhu · 2018
Cited alongside, same era.
On the convergence of a class of adam-type algorithms for non-convex optimization
X. Chen, S. Liu, R. Sun, and M. Hong · 2018
Cited alongside, same era.
Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator
C. Fang, C. J. Li, Z. Lin, and T. Zhang · 2018
Cited alongside, same era.
Revisiting small batch training for deep neural networks
D. Masters and C. Luschi · 2018
Cited alongside, same era.
Lower bounds for non-convex stochastic optimization
Y. Arjevani, Y. Carmon, J. C. Duchi, D. J. Foster, N. Srebro, and B. Woodworth · 2019
Later among the works it cites.
Momentum-based variance reduction in non-convex sgd
A. Cutkosky and F. Orabona · 2019
Later among the works it cites.
Stochastic model-based minimization of weakly convex functions
D. Davis and D. Drusvyatskiy · 2019
Later among the works it cites.
Spiderboost and momentum: Faster variance reduction algorithms
Z. Wang, K. Ji, Y. Zhou, Y. Liang, and V. Tarokh · 2019
Later among the works it cites.
Regularization matters: Generalization and optimization of neural nets vs their induced kernel
C. Wei, J. D. Lee, Q. Liu, and T. Ma · 2019
Later among the works it cites.
A stochastic composite gradient method with incremental variance reduction
J. Zhang and L. Xiao · 2019
Later among the works it cites.
Stochastic subgradient method converges on tame functions
D. Davis, D. Drusvyatskiy, S. Kakade, and J. D. Lee · 2020
Closest in time.
ProxSARAH: An efficient algorithmic framework for stochastic composite nonconvex optimization
N. H. Pham, L. M. Nguyen, D. T. Phan, and Q. Tran-Dinh · 2020
Closest in time.
Hybrid variance-reduced sgd algorithms for minimax problems with nonconvex-linear function
Q. Tran Dinh, D. Liu, and L. Nguyen · 2020
Closest in time.
Parallel and distributed asynchronous adaptive stochastic gradient methods
Y. Xu, Y. Xu, Y. Yan, C. Sutcher-Shepard, L. Grinberg, and J. Chen · 2020
Closest in time.
A hybrid stochastic optimization framework for composite nonconvex optimization
Q. Tran-Dinh, N. H. Pham, D. T. Phan, and L. M. Nguyen · 2021
Closest in time.
Katyusha acceleration for convex finite-sum compositional optimization
Y. Xu and Y. Xu · 2021
Closest in time.
Stochastic variance-reduced prox-linear algorithms for nonconvex composite optimization
J. Zhang and L. Xiao · 2021
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…