Fetching the paper…
Reading the bibliography…
Derivative-free optimization (DFO) has recently gained a lot of momentum in machine learning, spawning interest in the community to design faster methods for problems where gradients are not accessible.
Some methods of speeding up the convergence of iteration methods
Polyak, B. T · 1964
Earlier work this paper cites.
Random optimization
Matyas, J · 1965
Earlier work this paper cites.
A simplex method for function minimization
Nelder, J. A. and Mead, R · 1965
Earlier work this paper cites.
A method for solving the convex programming problem with convergence rate o (1/kˆ 2)
Nesterov, Y. E · 1983
Earlier work this paper cites.
Random gradient-free minimization of convex functions
Nesterov, Y. and Spokoiny, V · 2011
Earlier work this paper cites.
Regret analysis of stochastic and nonstochastic multi-armed bandit problems
Bubeck, S., Cesa-Bianchi, N., et al · 2012
Earlier work this paper cites.
An optimal method for stochastic composite optimization
Lan, G · 2012
Earlier work this paper cites.
Stochastic first-and zeroth-order methods for nonconvex stochastic programming
Ghadimi, S. and Lan, G · 2013
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
Johnson, R. and Zhang, T · 2013
Earlier work this paper cites.
Saga: A fast incremental gradient method with support for non-strongly convex composite objectives
Defazio, A., Bach, F., and Lacoste-Julien, S · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2014
Cited alongside, same era.
Introductory Lectures on Convex Optimization: A Basic Course
Nesterov, Y · 2014
Cited alongside, same era.
A universal catalyst for first-order optimization
Lin, H., Mairal, J., and Harchaoui, Z · 2015
Cited alongside, same era.
Katyusha: The first direct acceleration of stochastic gradient methods
Allen-Zhu, Z · 2017
Cited alongside, same era.
Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models
Chen, P.-Y., Zhang, H., Sharma, Y., Yi, J., and Hsieh, C.-J · 2017
Cited alongside, same era.
Sarah: A novel method for machine learning problems using stochastic recursive gradient, 2017
Stochastic heavy ball
Gadat, S., Panloup, F., Saadane, S., et al · 2018
Later among the works it cites.
An accelerated method for derivative-free smooth stochastic convex optimization
Gorbunov, E., Dvurechensky, P., and Gasnikov, A · 2018
Later among the works it cites.
An optimal randomized incremental gradient method
Lan, G. and Zhou, Y · 2018
Later among the works it cites.
Stochastic three points method for unconstrained smooth minimization
Bergou, E. H., Gorbunov, E., and Richtárik, P · 2019
Later among the works it cites.
A stochastic derivative free optimization method with momentum
Gorbunov, E., Bibi, A., Sener, O., Bergou, E. H., and Richtárik, P · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Nguyen, L. M., Liu, J., Scheinberg, K., and Takáč, M · 2017
Cited alongside, same era.
Evolution strategies as a scalable alternative to reinforcement learning
Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I · 2017
Cited alongside, same era.
Fast Stochastic Variance Reduced Gradient Method with Momentum Acceleration for Machine Learning
Shang, F., Liu, Y., Cheng, J., and Zhuo, J · 2017
Cited alongside, same era.
Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator
Fang, C., Li, C. J., Lin, Z., and Zhang, T · 2018
Cited alongside, same era.
Stochastic zeroth-order optimization via variance reduction method
Liu, L., Cheng, M., Hsieh, C.-J., and Tao, D
Cited in the paper.
Zeroth-order stochastic variance reduction for nonconvex optimization
Liu, S., Kailkhura, B., Chen, P.-Y., Ting, P., Chang, S., and Amini, L
Cited in the paper.
Improved zeroth-order variance reduced algorithms and analysis for nonconvex optimization
Ji, K., Wang, Z., Zhou, Y., and Liang, Y · 2019
Later among the works it cites.
A unified variance-reduced accelerated gradient method for convex optimization
Lan, G., Li, Z., and Zhou, Y · 2019
Later among the works it cites.
The role of memory in stochastic optimization
Orvieto, A., Kohler, J., and Lucchi, A · 2019
Later among the works it cites.
Stochastic nested variance reduction for nonconvex optimization
Zhou, D., Xu, P., and Gu, Q · 2020
Closest in time.