Fetching the paper…
Reading the bibliography…
We present a uniform analysis of biased stochastic gradient methods for minimizing convex, strongly convex, and non-convex composite objectives, and identify settings where bias is useful in stochastic gradient estimation.
A stochastic approximation method
Robbins, H., and Monro, S · 1951
Earlier work this paper cites.
Splitting algorithms for the sum of two nonlinear operators
Lions, P. L., and Mercier, B · 1979
Earlier work this paper cites.
Introductory lectures on convex programming
Nesterov, Y · 2004
Earlier work this paper cites.
Pattern recognition and machine learning
Bishop, C. M · 2006
Earlier work this paper cites.
A fast iterative shrinkage-thresholding algorithm for linear inverse problems
Beck, A., and Teboulle, M · 2009
Earlier work this paper cites.
A stochastic gradient method with an exponential convergence rate for finite training sets
Roux, N. L., Schmidt, M., and Bach, F. R · 2012
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
Johnson, R., and Zhang, T · 2013
Earlier work this paper cites.
Stochastic dual coordinate ascent methods for regularized loss minimization
Shalev-Shwartz, S., and Zhang, T · 2013
Earlier work this paper cites.
SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives
Defazio, A., Bach, F., and Lacoste-Julien, S · 2014
Earlier work this paper cites.
Finito: A faster, permutable incremental gradient method for big data problems
Defazio, A., Caetano, T., and Domke, J · 2014
Earlier work this paper cites.
Incremental majorization-minimization optimization with application to large-scale machine learning
Mairal, J · 2014
Earlier work this paper cites.
A proximal stochastic gradient method with progressive variance reduction
Xiao, L., and Zhang, T · 2014
Earlier work this paper cites.
Faster and simple PCA via convex optimization
Garber, D., and Hazan, E · 2015
Cited alongside, same era.
Variance reduced stochastic gradient descent with neighbors
Hofmann, T., Lucchi, A., Lacoste-Julien, S., and McWilliams, B · 2015
Cited alongside, same era.
Variance reduction for faster non-convex optimization
Allen-Zhu, Z., and Hazan, E · 2016
Cited alongside, same era.
A simple practical accelerated method for finite sums
Defazio, A · 2016
Cited alongside, same era.
Stochastic variance reduction for nonconvex optimization
Reddi, S. J., Hefny, A., Sra, S., Póczos, B., and Smola, A · 2016
Cited alongside, same era.
Fast stochastic methods for nonsmooth nonconvex optimization
Reddi, S. J., Sra, S., Póczos, B., and Smola, A · 2016
Natasha 2: Faster non-convex optimization than SGD
Allen-Zhu, Z · 2018
Later among the works it cites.
Improved SVRG for non-strongly-convex or sum-of-non-convex objectives
Allen-Zhu, Z., and Yuan, Y · 2018
Later among the works it cites.
Optimization methods for large-scale machine learning
Bottou, L., Curtis, F. E., , and Nocedal, J · 2018
Later among the works it cites.
Mathematical Image Processing
Bredies, K., and Lorenz, D · 2018
Later among the works it cites.
Stochastic primal-dual hybrid gradient algorithm with arbitrary sampling and imaging applications
Chambolle, A., Ehrhardt, M. J., Richtárik, P., and Schönlieb, C.-B · 2018
Later among the works it cites.
Spider: Near-optimal non-convex optimization via stochastic path integrated differential estimator
Fang, C., Li, C. J., Lin, Z., and Zhang, T · 2018
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Natasha: Faster non-convex stochastic optimization via strongly non-convex parameter
Allen-Zhu, Z · 2017
Cited alongside, same era.
Activity identification and local linear convergence of Forward–Backward-type methods
Liang, J., Fadili, J., and Peyré, G · 2017
Cited alongside, same era.
SARAH: A novel method for machine learning problems using stochastic recursive gradient
Nguyen, L. M., Liu, J., Scheinberg, K., and Takáĉ, M · 2017
Cited alongside, same era.
Minimizing finite sums with the stochastic average gradient
Schmidt, M., Roux, N. L., and Bach, F · 2017
Cited alongside, same era.
Katyusha: The first direct acceleration of stochastic gradient methods
Allen-Zhu, Z · 2018
Cited alongside, same era.
Katyusha X: Practical momentum method for stochastic sum-of-nonconvex optimization
Allen-Zhu, Z · 2018
Cited alongside, same era.
Later among the works it cites.
ASVRG: Accelerated proximal SVRG
Shang, F., Jiao, L., Zhou, K., Cheng, J., Ren, Y., and Jin, Y · 2018
Later among the works it cites.
SpiderBoost: A class of faster variance-reduced algorithms for nonconvex optimization
Wang, Z., Ji, K., Zhou, Y., Liang, Y., and Tarokh, V · 2018
Later among the works it cites.
A simple stochastic variance reduced algorithm with fast convergence rates
Zhou, K., Shang, F., and Cheng, J · 2018
Later among the works it cites.
A unified variance-reduced accelerated gradient method for convex optimization
Lan, G., Li, Z., and Zhou, Y · 2019
Closest in time.
ProxSARAH: An efficient algorithmic framework for stochastic composite nonconvex optimization
Pham, N. H., Nguyen, L. M., Phan, D. T., and Tran-Dinh, Q · 2019
Closest in time.
Momentum schemes with stochastic variance reduction for nonconvex composite optimization
Zhou, Y., Wang, Z., Ji, K., Liang, Y., and Tarokh, V · 2019
Closest in time.