Fetching the paper…
Reading the bibliography…
We develop a class of algorithms, as variants of the stochastically controlled stochastic gradient (SCSG) methods (Lei and Jordan, 2016), for the smooth non-convex finite-sum optimization problem.
Gradient methods for minimizing functionals
Boris Teodorovich Polyak · 1963
Earlier work this paper cites.
Generalized Linear Models
Peter McCullagh and John A Nelder · 1989
Earlier work this paper cites.
Convergence properties of backpropagation for neural nets via theory of stochastic gradient methods. Part 1
Alexei A Gaivoronski · 1994
Earlier work this paper cites.
A new class of incremental gradient methods for least squares problems
Dimitri P Bertsekas · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
An incremental gradient (-projection) method with momentum term and adaptive stepsize rule
Paul Tseng · 1998
Earlier work this paper cites.
Introductory Lectures on Convex Optimization: A Basic Course
Yurii Nesterov · 2004
Earlier work this paper cites.
Robust stochastic approximation approach to stochastic programming
Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro · 2009
Earlier work this paper cites.
Adaptive subgradient methods for online learning and stochastic optimization
John Duchi, Elad Hazan, and Yoram Singer · 2011
Earlier work this paper cites.
Stochastic first-and zeroth-order methods for nonconvex stochastic programming
Saeed Ghadimi and Guanghui Lan · 2013
Cited alongside, same era.
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George E Dahl, and Geoffrey E Hinton · 2013
Cited alongside, same era.
A lower bound for the optimization of finite sums
Alekh Agarwal and Leon Bottou · 2014
Cited alongside, same era.
Linear coupling: An ultimate unification of gradient and mirror descent
Zeyuan Allen-Zhu and Lorenzo Orecchia · 2014
Cited alongside, same era.
Adam: A method for stochastic optimization
Diederik Kingma and Jimmy Ba · 2014
Cited alongside, same era.
Variance reduction for faster non-convex optimization
Zeyuan Allen-Zhu and Elad Hazan · 2016
Later among the works it cites.
Accelerated methods for non-convex optimization
Yair Carmon, John C Duchi, Oliver Hinder, and Aaron Sidford · 2016
Later among the works it cites.
Accelerated gradient methods for nonconvex nonlinear and stochastic programming
Saeed Ghadimi and Guanghui Lan · 2016
Later among the works it cites.
Linear convergence of gradient and proximal-gradient methods under the Polyak-Lojasiewicz condition
Hamed Karimi, Julie Nutini, and Mark Schmidt · 2016
Later among the works it cites.
Less than a single pass: Stochastically controlled stochastic gradient method
Lihua Lei and Michael I Jordan · 2016
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Zeyuan Allen-Zhu and Yang Yuan · 2015
Cited alongside, same era.
Stop wasting my gradients: Practical SVRG
Reza Harikandeh, Mohamed Osama Ahmed, Alim Virani, Mark Schmidt, Jakub Konečnỳ, and Scott Sallinen · 2015
Cited alongside, same era.
Deep learning
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton · 2015
Cited alongside, same era.
Finding approximate local minima for nonconvex optimization in linear time
Naman Agarwal, Zeyuan Allen-Zhu, Brian Bullins, Elad Hazan, and Tengyu Ma · 2016
Cited alongside, same era.
Later among the works it cites.
Stochastic variance reduction for nonconvex optimization
Sashank J Reddi, Ahmed Hefny, Suvrit Sra, Barnabas Poczos, and Alex Smola · 2016
Later among the works it cites.
Fast incremental method for nonconvex optimization
Sashank J Reddi, Suvrit Sra, Barnabás Póczos, and Alex Smola · 2016
Later among the works it cites.
Natasha: Faster stochastic non-convex optimization via strongly non-convex parameter
Zeyuan Allen-Zhu · 2017
Closest in time.
Convex until proven guilty: Dimension-free acceleration of gradient descent on non-convex functions
Yair Carmon, Oliver Hinder, John C Duchi, and Aaron Sidford · 2017
Closest in time.