Fetching the paper…
Reading the bibliography…
We propose ZeroSARAH -- a novel variant of the variance-reduced method SARAH (Nguyen et al., 2017) -- for minimizing the average of a large number of nonconvex functions $\frac{1}{n}\sum_{i=1}^{n}f_i(x)$.
Introductory Lectures on Convex Optimization: A Basic Course
Yurii Nesterov · 2004
Earlier work this paper cites.
From convex to nonconvex: a loss function analysis for binary classification
Lei Zhao, Musa Mammadov, and John Yearwood · 2010
Earlier work this paper cites.
LIBSVM: a library for support vector machines
Chih-Chung Chang and Chih-Jen Lin · 2011
Earlier work this paper cites.
Stochastic first-and zeroth-order methods for nonconvex stochastic programming
Saeed Ghadimi and Guanghui Lan · 2013
Earlier work this paper cites.
Understanding machine learning: from theory to algorithms
Shai Shalev-Shwartz and Shai Ben-David · 2014
Earlier work this paper cites.
Variance reduction for faster non-convex optimization
Zeyuan Allen-Zhu and Elad Hazan · 2016
Earlier work this paper cites.
Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
Saeed Ghadimi, Guanghui Lan, and Hongchao Zhang · 2016
Earlier work this paper cites.
Stochastic variance reduction for nonconvex optimization
Sashank J Reddi, Ahmed Hefny, Suvrit Sra, Barnabás Póczos, and Alex Smola · 2016
Earlier work this paper cites.
Non-convex optimization for machine learning
Prateek Jain and Purushottam Kar · 2017
Earlier work this paper cites.
Non-convex finite-sum optimization via SCSG methods
Lihua Lei, Cheng Ju, Jianbo Chen, and Michael I Jordan · 2017
Earlier work this paper cites.
SARAH: A novel method for machine learning problems using stochastic recursive gradient
Lam M Nguyen, Jie Liu, Katya Scheinberg, and Martin Takáč · 2017
Cited alongside, same era.
SPIDER: Near-optimal non-convex optimization via stochastic path-integrated differential estimator
Cong Fang, Chris Junchi Li, Zhouchen Lin, and Tong Zhang · 2018
Cited alongside, same era.
A simple proximal stochastic gradient method for nonsmooth nonconvex optimization
Zhize Li and Jian Li · 2018
Cited alongside, same era.
SpiderBoost and momentum: Faster stochastic variance reduction algorithms
Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, and Vahid Tarokh · 2018
Cited alongside, same era.
Stochastic nested variance reduction for nonconvex optimization
Dongruo Zhou, Pan Xu, and Quanquan Gu · 2018
Cited alongside, same era.
Convergence of distributed stochastic variance reduced methods without sampling extra data
Shicong Cen, Huishuai Zhang, Yuejie Chi, Wei Chen, and Tie-Yan Liu · 2020
Later among the works it cites.
Adaptivity of stochastic gradient methods for nonconvex optimization
Samuel Horváth, Lihua Lei, Peter Richtárik, and Michael I Jordan · 2020
Later among the works it cites.
SCAFFOLD: Stochastic controlled averaging for federated learning
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh · 2020
Later among the works it cites.
Better theory for SGD in the nonconvex world
Ahmed Khaled and Peter Richtárik · 2020
Later among the works it cites.
A unified analysis of stochastic gradient methods for nonconvex federated optimization
Zhize Li and Peter Richtárik · 2020
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Stabilized SVRG: Simple variance reduction for nonconvex optimization
Rong Ge, Zhize Li, Weiyao Wang, and Xiang Wang · 2019
Cited alongside, same era.
SSRGD: Simple stochastic recursive gradient descent for escaping saddle points
Zhize Li · 2019
Cited alongside, same era.
ProxSARAH: An efficient algorithmic framework for stochastic composite nonconvex optimization
Nhan H Pham, Lam M Nguyen, Dzung T Phan, and Quoc Tran-Dinh · 2019
Cited alongside, same era.
Hybrid stochastic gradient descent algorithms for stochastic nonconvex optimization
Quoc Tran-Dinh, Nhan H Pham, Dzung T Phan, and Lam M Nguyen · 2019
Cited alongside, same era.
Later among the works it cites.
Improving the sample and communication complexity for decentralized non-convex optimization: Joint gradient estimation and tracking
Haoran Sun, Songtao Lu, and Mingyi Hong · 2020
Later among the works it cites.
A short note of PAGE: Optimal convergence rates for nonconvex optimization
Zhize Li · 2021
Closest in time.
PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization
Zhize Li, Hongyan Bao, Xiangliang Zhang, and Peter Richtárik · 2021
Closest in time.
FedPAGE: A fast local stochastic gradient method for communication-efficient federated learning
Haoyu Zhao, Zhize Li, and Peter Richtárik · 2021
Closest in time.