Fetching the paper…
Reading the bibliography…
In this paper, we propose a novel sufficient decrease technique for variance reduced stochastic gradient descent methods such as SAG, SVRG and SAGA.
Convergence conditions for ascent methods
P. Wolfe · 1969
Earlier work this paper cites.
Line search algorithms with guaranteed sufficient decrease
J. More and D. Thuente · 1994
Earlier work this paper cites.
De-noising by soft-thresholding
D. L. Donoho · 1995
Earlier work this paper cites.
Introductory Lectures on Convex Optimization: A Basic Course
Y. Nesterov · 2004
Earlier work this paper cites.
Solving large scale linear prediction problems using stochastic gradient descent algorithms
T. Zhang · 2004
Earlier work this paper cites.
ImageNet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
An optimal method for stochastic composite optimization
G. Lan · 2012
Earlier work this paper cites.
A stochastic gradient method with an exponential convergence rate for finite training sets
N. Le Roux, M. Schmidt, and F. Bach · 2012
Earlier work this paper cites.
Non-strongly-convex smooth stochastic approximation with convergence rate O ( 1 / n ) {O}(1/n)
F. Bach and E. Moulines · 2013
Earlier work this paper cites.
Advanced topics in machine learning part II: 5. Proximal methods
L. Baldassarre and M. Pontil · 2013
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
R. Johnson and T. Zhang · 2013
Earlier work this paper cites.
Minimizing finite sums with the stochastic average gradient
M. Schmidt, N. Le Roux, and F. Bach · 2013
Earlier work this paper cites.
Stochastic dual coordinate ascent methods for regularized loss minimization
S. Shalev-Shwartz and T. Zhang · 2013
Earlier work this paper cites.
Linear convergence with condition number independent access of full gradients
L. Zhang, M. Mahdavi, and R. Jin · 2013
Cited alongside, same era.
SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives
A. Defazio, F. Bach, and S. Lacoste-Julien · 2014
Cited alongside, same era.
Finito: A faster, permutable incremental gradient method for big data problems
A. Defazio, T. Caetano, and J. Domke · 2014
Cited alongside, same era.
Stochastic proximal gradient descent with acceleration techniques
A. Nitanda · 2014
Cited alongside, same era.
A proximal stochastic gradient method with progressive variance reduction
L. Xiao and T. Zhang · 2014
Cited alongside, same era.
Stop wasting my gradients: Practical SVRG
R. Babanezhad, M. O. Ahmed, A. Virani, M. Schmidt, J. Konecny, and S. Sallinen · 2015
Incremental majorization-minimization optimization with application to large-scale machine learning
J. Mairal · 2015
Later among the works it cites.
On variance reduction in stochastic gradient descent and its asynchronous variants
S. Reddi, A. Hefny, S. Sra, B. Poczos, and A. Smola · 2015
Later among the works it cites.
Stochastic optimization with importance sampling for regularized loss minimization
P. Zhao and T. Zhang · 2015
Later among the works it cites.
Variance reduction for faster non-convex optimization
Z. Allen-Zhu and E. Hazan · 2016
Later among the works it cites.
Improved SVRG for non-strongly-convex or sum-of-non-convex objectives
Z. Allen-Zhu and Y. Yuan · 2016
Later among the works it cites.
Variance-reduced and projection-free stochastic optimization
E. Hazan and H. Luo · 2016
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization
R. Frostig, R. Ge, S. M. Kakade, and A. Sidford · 2015
Cited alongside, same era.
Variance reduced stochastic gradient descent with neighbors
T. Hofmann, A. Lucchi, S. Lacoste-Julien, and B. McWilliams · 2015
Cited alongside, same era.
Adam: A method for stochastic optimization
D. P. Kingma and J. Ba · 2015
Cited alongside, same era.
Accelerated proximal gradient methods for nonconvex programming
H. Li and Z. Lin · 2015
Cited alongside, same era.
A universal catalyst for first-order optimization
H. Lin, J. Mairal, and Z. Harchaoui · 2015
Cited alongside, same era.
Probabilistic line searches for stochastic optimization
M. Mahsereci and P. Hennig · 2015
Cited alongside, same era.
Later among the works it cites.
J. D. Lee, Q. Lin, T. Ma, and T. Yang · 2016
Later among the works it cites.
Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization
S. J. Reddi, S. Sra, B. Poczos, and A. Smola · 2016
Later among the works it cites.
Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization
S. Shalev-Shwartz and T. Zhang · 2016
Later among the works it cites.
Without-replacement sampling for stochastic gradient methods
O. Shamir · 2016
Later among the works it cites.
Katyusha: The first direct acceleration of stochastic gradient methods
Z. Allen-Zhu · 2017
Closest in time.
SARAH: A novel method for machine learning problems using stochastic recursive gradient
L. Nguyen, J. Liu, K. Scheinberg, and M. Takáč · 2017
Closest in time.