Fetching the paper…
Reading the bibliography…
Recent years have witnessed exciting progress in the study of stochastic variance reduced gradient methods (e.g., SVRG, SAGA), their accelerated variants (e.g, Katyusha) and their extensions in many different settings (e.g., online, sparse, asynchronous, distributed).
A stochastic approximation method
Robbins, H. and Monro, S · 1951
Earlier work this paper cites.
Introductory Lectures on Convex Optimization: A Basic Course
Nesterov, Y · 2004
Earlier work this paper cites.
Hogwild: A lock-free approach to parallelizing stochastic gradient descent
Recht, B., Re, C., Wright, S., and Niu, F · 2011
Earlier work this paper cites.
A stochastic gradient method with an exponential convergence rate for finite training sets
Roux, N. L., Schmidt, M., and Bach, F · 2012
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
Johnson, R. and Zhang, T · 2013
Earlier work this paper cites.
Stochastic dual coordinate ascent methods for regularized loss minimization
Shalev-Shwartz, S. and Zhang, T · 2013
Earlier work this paper cites.
Linear convergence with condition number independent access of full gradients
Zhang, L., Mahdavi, M., and Jin, R · 2013
Earlier work this paper cites.
SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives
Defazio, A., Bach, F., and Lacoste-Julien, S · 2014
Earlier work this paper cites.
An accelerated proximal coordinate gradient method
Lin, Q., Lu, Z., and Xiao, L · 2014
Earlier work this paper cites.
Stochastic proximal gradient descent with acceleration techniques
Nitanda, A · 2014
Earlier work this paper cites.
A proximal stochastic gradient method with progressive variance reduction
Xiao, L. and Zhang, T · 2014
Cited alongside, same era.
A universal catalyst for first-order optimization
Lin, H., Mairal, J., and Harchaoui, Z · 2015
Cited alongside, same era.
On variance reduction in stochastic gradient descent and its asynchronous variants
Reddi, S., Hefny, A., Sra, S., Poczos, B., and Smola, A · 2015
Cited alongside, same era.
Stochastic primal-dual coordinate method for regularized empirical risk minimization
Zhang, Y. and Xiao, L · 2015
Cited alongside, same era.
A simple practical accelerated method for finite sums
Defazio, A · 2016
Cited alongside, same era.
Mini-batch semi-stochastic gradient descent in the proximal setting
Konečný, J., Liu, J., Richtárik, P., , and Takáč, M · 2016
Cited alongside, same era.
Linear coupling: An ultimate unification of gradient and mirror descent
Allen-Zhu, Z. and Orecchia, L · 2017
Later among the works it cites.
Accelerated stochastic mirror descent algorithms for composite non-strongly convex optimization
Hien, L., Lu, C., Xu, H., and Feng, J · 2017
Later among the works it cites.
ASAGA: Asynchronous parallel SAGA
Leblond, R., Pedregosa, F., and Lacoste-Julien, S · 2017
Later among the works it cites.
Distributed stochastic variance reduced gradient methods by sampling extra data with replacement
Lee, J. D., Lin, Q., Ma, T., and Yang, T · 2017
Later among the works it cites.
Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent
Lian, X., Zhang, C., Zhang, H., Hsieh, C.-J., Zhang, W., and Liu, J · 2017
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
Shalev-Shwartz, S. and Zhang, T · 2016
Cited alongside, same era.
Homotopy smoothing for non-smooth problems with lower complexity than O ( 1 / ϵ ) {O}(1/\epsilon)
Xu, Y., Yan, Y., Lin, Q., and Yang, T · 2016
Cited alongside, same era.
Katyusha: The first direct acceleration of stochastic gradient methods
Allen-Zhu, Z · 2017
Cited alongside, same era.
Optimal black-box reductions between optimization objectives
Allen-Zhu, Z. and Hazan, E
Cited in the paper.
Variance reduction for faster non-convex optimization
Allen-Zhu, Z. and Hazan, E
Cited in the paper.
Perturbed iterate analysis for asynchronous stochastic optimization
Mania, H., Pan, X., Papailiopoulos, D., Recht, B., Ramchandran, K., and Jordan, M. I · 2017
Later among the works it cites.
Breaking the nonsmooth barrier: A scalable parallel method for composite optimization
Pedregosa, F., Leblond, R., and Lacoste-Julien, S · 2017
Later among the works it cites.
Minimizing finite sums with the stochastic average gradient
Schmidt, M., Roux, N. L., and Bach, F · 2017
Later among the works it cites.
Online variance reduction for stochastic optimization
Borsos, Z., Krause, A., and Levy, K. Y · 2018
Closest in time.