Fetching the paper…
Reading the bibliography…
One of the major issues in stochastic gradient descent (SGD) methods is how to choose an appropriate step size while running the algorithm.
Accelerated stochastic approximation
H. Kesten · 1958
Earlier work this paper cites.
Two-point step size gradient methods
J. Barzilai and J. M. Borwein · 1988
Earlier work this paper cites.
Acceleration of stochastic approximation by averaging
B. T. Polyak and A. B. Juditsky · 1992
Earlier work this paper cites.
On the Barzilai and Borwein choice of steplength for the gradient method
M. Raydan · 1993
Earlier work this paper cites.
The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem
M. Raydan · 1997
Earlier work this paper cites.
Introductory lectures on convex optimization
Y. Nesterov · 2004
Earlier work this paper cites.
Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming
Y.-H. Dai and R. Fletcher · 2005
Earlier work this paper cites.
On the Barzilai-Borwein method
R. Fletcher · 2005
Earlier work this paper cites.
The cyclic Barzilai-Borwein method for unconstrained optimization
Y.-H. Dai, W. W. Hager, K. Schittkowski, and H. Zhang · 2006
Earlier work this paper cites.
A stochastic quasi-newton method for online convex optimization
N. N. Schraudolph, J. Yu, and S. Günter · 2007
Earlier work this paper cites.
Projected Barzilai-Borwein methods for large scale nonnegative image restorations
Y. Wang and S. Ma · 2007
Cited alongside, same era.
Sparse reconstruction by separable approximation
S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo · 2009
Cited alongside, same era.
A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation
Z. Wen, W. Yin, D. Goldfarb, and Y. Zhang · 2010
Cited alongside, same era.
Adaptive subgradient methods for online learning and stochastic optimization
J. Duchi, E. Hazan, and Y. Singer · 2011
Cited alongside, same era.
A stochastic gradient method with an exponential convergence rate for finite training sets
R. L. Roux, M. Schmidt, and F. Bach · 2012
Cited alongside, same era.
A new analysis on the Barzilai-Borwein gradient method
Y.-H. Dai · 2013
Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm
D. Needell, N. Srebro, and R. Ward · 2014
Later among the works it cites.
Stochastic proximal gradient descent with acceleration techniques
A. Nitanda · 2014
Later among the works it cites.
A proximal stochastic gradient method with progressive variance reduction
L. Xiao and T. Zhang · 2014
Later among the works it cites.
Stop wasting my gradients: Practical SVRG
R. Babanezhad, M. O. Ahmed, A. Virani, M. Schmidt, K. Konečnỳ, and S. Sallinen · 2015
Later among the works it cites.
Probabilistic line searches for stochastic optimization
M. Mahsereci and P. Hennig · 2015
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Accelerating stochastic gradient descent using predictive variance reduction
R. Johnson and T. Zhang · 2013
Cited alongside, same era.
Semi-stochastic gradient descent methods
J. Konečnỳ and P. Richtárik · 2013
Cited alongside, same era.
Stochastic dual coordinate ascent methods for regularized loss minimization
S. Shalev-Shwartz and T. Zhang · 2013
Cited alongside, same era.
SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives
A. Defazio, F. Bach, and S. Lacoste-Julien · 2014
Cited alongside, same era.
P. Y. Massé and Y. Ollivier · 2015
Later among the works it cites.
Stochastic gradient descent with Barzilai-Borwein update step for svm
K. Sopyła and P. Drozda · 2015
Later among the works it cites.
Stochastic optimization with importance sampling for regularized loss minimization
P. Zhao and T. Zhang · 2015
Later among the works it cites.
Variance reduction for faster non-convex optimization
Z. Allen-Zhu and E. Hazan · 2016
Closest in time.
Stochastic variance reduction for nonconvex optimization
S. J. Reddi, A. Hefny, S. Sra, B. Poczos, and A. Smola · 2016
Closest in time.