Fetching the paper…
Reading the bibliography…
We introduce a new approach to develop stochastic optimization algorithms for a class of stochastic composite and possibly nonconvex optimization problems.
A stochastic approximation method
Herbert Robbins and Sutton Monro · 1951
Earlier work this paper cites.
Simplified neuron model as a principal component analyzer
E. Oja · 1982
Earlier work this paper cites.
Problem Complexity and Method Efficiency in Optimization
A. Nemirovskii and D. Yudin · 1983
Earlier work this paper cites.
Acceleration of stochastic approximation by averaging
B. Polyak and A. Juditsky · 1992
Earlier work this paper cites.
Online learning and stochastic approximations
Léon Bottou · 1998
Earlier work this paper cites.
Introductory lectures on convex optimization: A basic course
Y. Nesterov · 2004
Earlier work this paper cites.
Cubic regularization of Newton method and its global performance
Y. Nesterov and B.T. Polyak · 2006
Earlier work this paper cites.
Robust stochastic approximation approach to stochastic programming
A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro · 2009
Earlier work this paper cites.
Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization
A. Agarwal, P. L. Bartlett, P. Ravikumar, and M. J. Wainwright · 2010
Earlier work this paper cites.
Large-scale machine learning with stochastic gradient descent
L. Bottou · 2010
Earlier work this paper cites.
From convex to nonconvex: a loss function analysis for binary classification
L. Zhao, M. Mammadov, and J. Yearwood · 2010
Earlier work this paper cites.
Incremental proximal methods for large scale convex optimization
D.P. Bertsekas · 2011
Earlier work this paper cites.
LIBSVM: A library for Support Vector Machines
C.-C. Chang and C.-J. Lin · 2011
Earlier work this paper cites.
Adaptive subgradient methods for online learning and stochastic optimization
John Duchi, Elad Hazan, and Yoram Singer · 2011
Earlier work this paper cites.
Non-asymptotic analysis of stochastic approximation algorithms for machine learning
Eric Moulines and Francis R Bach · 2011
Earlier work this paper cites.
Stochastic first-and zeroth-order methods for nonconvex stochastic programming
S. Ghadimi and G. Lan · 2013
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
R. Johnson and T. Zhang · 2013
Earlier work this paper cites.
Stochastic dual coordinate ascent methods for regularized loss minimization
S. Shalev-Shwartz and T. Zhang · 2013
Earlier work this paper cites.
SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives
A. Defazio, F. Bach, and S. Lacoste-Julien · 2014
Earlier work this paper cites.
Finito: A faster, permutable incremental gradient method for big data problems
A. Defazio, T. Caetano, and J. Domke · 2014
Earlier work this paper cites.
ADAM: A Method for Stochastic Optimization
D. P. Kingma and J. Ba · 2014
Earlier work this paper cites.
A lower bound for the optimization of finite sums
A. Agarwal and L. Bottou · 2015
Earlier work this paper cites.
Convergence rates of sub-sampled Newton methods
Murat A Erdogdu and Andrea Montanari · 2015
Earlier work this paper cites.
Escaping from saddle points - online stochastic gradient for tensor decomposition
R. Ge, F. Huang, C. Jin, and Y. Yuan · 2015
Earlier work this paper cites.
Deep learning
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton · 2015
Earlier work this paper cites.
A universal catalyst for first-order optimization
H. Lin, J. Mairal, and Z. Harchaoui · 2015
Earlier work this paper cites.
Incremental majorization-minimization optimization with application to large-scale machine learning
J. Mairal · 2015
Earlier work this paper cites.
Stochastic optimization with importance sampling for regularized loss minimization
P. Zhao and T. Zhang · 2015
Cited alongside, same era.
Improved SVRG for Non-Strongly-Convex or Sum-of-Non-Convex Objectives
Zeyuan Allen-Zhu and Yang Yuan · 2016
Cited alongside, same era.
A stochastic quasi-Newton method for large-scale optimization
Richard H Byrd, SL Hansen, Jorge Nocedal, and Yoram Singer · 2016
Cited alongside, same era.
Accelerated gradient methods for nonconvex nonlinear and stochastic programming
S. Ghadimi and G. Lan · 2016
Cited alongside, same era.
Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
S. Ghadimi, G. Lan, and H. Zhang · 2016
Cited alongside, same era.
Deep learning
I. Goodfellow, Y. Bengio, and A. Courville · 2016
Cited alongside, same era.
When does stochastic gradient algorithm work well?
L. M. Nguyen, N. H. Nguyen, D. T. Phan, J. R. Kalagnanam, and K. Scheinberg · 2018
Later among the works it cites.
Inexact SARAH Algorithm for Stochastic Optimization
L. M. Nguyen, K. Scheinberg, and M. Takac · 2018
Later among the works it cites.
Adaptive stochastic variance reduction for subsampled Newton method with cubic regularization
J. Zhang, L. Xiao, and S. Zhang · 2018
Later among the works it cites.
Stochastic nested variance reduction for nonconvex optimization
D. Zhou, P. Xu, and Q. Gu · 2018
Later among the works it cites.
A simple stochastic variance reduced algorithm with fast convergence rates
K. Zhou, F. Shang, and J. Cheng · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Linear convergence of gradient and proximal-gradient methods under the Polyak-łojasiewicz condition
H. Karimi, J. Nutini, and M. Schmidt · 2016
Cited alongside, same era.
Mini-batch semi-stochastic gradient descent in the proximal setting
Jakub Konečný, Jie Liu, Peter Richtárik, and Martin Takáč · 2016
Cited alongside, same era.
Stochastic Frank-Wolfe methods for nonconvex optimization
S. J. Reddi, S. Sra, B. Póczos, and A. Smola · 2016
Cited alongside, same era.
Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization
S. J. Reddi, S. Sra, B. Póczos, and A. J. Smola · 2016
Cited alongside, same era.
Tight complexity bounds for optimizing composite objectives
B. E. Woodworth and N. Srebro · 2016
Cited alongside, same era.
Katyusha: The first direct acceleration of stochastic gradient methods
Z. Allen-Zhu · 2017
Cited alongside, same era.
Lower bounds for non-convex stochastic optimization
Y. Arjevani, Y. Carmon, J. C. Duchi, D. J. Foster, N. Srebro, and B. Woodworth · 2019
Closest in time.
Exact and Inexact Subsampled Newton Methods for Optimization
R. Bollapragada, R. Byrd, and J. Nocedal · 2019
Closest in time.
Momentum-based variance reduction in non-convex SGD
A. Cutkosky and F. Orabona · 2019
Closest in time.
Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems
D. Davis and B. Grimmer · 2019
Closest in time.
On the bias-variance tradeoff in stochastic gradient methods
D. Driggs, J. Liang, and C.-B. Schönlieb · 2019
Closest in time.
Sharp Analysis for Nonconvex SGD Escaping from Saddle Points
C. Fang, Z. Lin, and T. Zhang · 2019
Closest in time.
The complexity of making the gradient small in stochastic convex optimization
D. Foster, A. Sekhari, O. Shamir, N. Srebro, K. Sridharan, and B. Woodworth · 2019
Closest in time.
Stabilized SVRG: Simple variance reduction for nonconvex optimization
R. Ge, Z. Li, W. Wang, and X. Wang · 2019
Closest in time.
On variance reduction for stochastic smooth convex optimization with multiplicative noise
A. Jofré and P. Thompson · 2019
Closest in time.
Don’t jump through hoops and remove those loops: SVRG and Katyusha are better without the outer loop
D. Kovalev, S. Horvath, and P. Richtarik · 2019
Closest in time.
SSRGD: Simple stochastic recursive gradient descent for escaping saddle points
Z. Li · 2019
Closest in time.
Simple stochastic gradient methods for non-smooth non-convex regularized optimization
M. Metel and A. Takeda · 2019
Closest in time.
Optimal finite-sum smooth non-convex optimization with SARAH
L. M. Nguyen, M. van Dijk, D. T. Phan, P. H. Nguyen, T.-W. Weng, and J. R. Kalagnanam · 2019
Closest in time.
Sub-sampled Newton methods I: Globally convergent algorithms
F. Roosta-Khorasani and M. W. Mahoney · 2019
Closest in time.
A Representer Theorem for Deep Neural Networks
M. Unser · 2019
Closest in time.
SpiderBoost and Momentum: Faster Variance Reduction Algorithms
Z. Wang, K. Ji, Y. Zhou, Y. Liang, and V. Tarokh · 2019
Closest in time.
Stochastic variance-reduced cubic regularization for nonconvex optimization
Z. Wang, Y. Zhou, Y. Liang, and G. Lan · 2019
Closest in time.
Lower bounds for smooth nonconvex finite-sum optimization
D. Zhou and Q. Gu · 2019
Closest in time.
Stochastic recursive variance-reduced cubic regularization methods
D. Zhou and Q. Gu · 2019
Closest in time.
Momentum schemes with stochastic variance reduction for nonconvex composite optimization
Y. Zhou, Z. Wang, K. Ji, Y. Liang, and V. Tarokh · 2019
Closest in time.
ProxSARAH: An efficient algorithmic framework for stochastic composite nonconvex optimization
H. N. Pham, M. L. Nguyen, T. D. Phan, and Q. Tran-Dinh · 2020
Closest in time.