Heuristic adaptive fast gradient method in stochastic optimization tasks

‘‘Griewank A. On automatic differentiation // Mathematical Programming: recent developments and applications. 1989. V. 6. № 6. P. 83–107.’’

1989

Earlier work this paper cites.

‘‘LeCun Y., Bottou L., Bengio Y., Haffner P. Gradient-based learning applied to document recognition // Proceedings of the IEEE. 1998. V. 86. № 11. P. 2278–2324.’’

1998

Earlier work this paper cites.

‘‘Nocedal J., Wright S. Numerical optimization. Springer Science & Business Media. 2006.’’

2006

Earlier work this paper cites.

‘‘Gupta M. D., Huang T. Bregman distance to l1 regularized logistic regression // ICPR, 2008.’’

2008

Earlier work this paper cites.

‘‘Juditsky A., Nemirovski A. Large deviations of vector-valued martingales in 2–smooth normed spaces // e-print. arXiv:0809.0813. 2008.’’

Original

2008

Earlier work this paper cites.

‘‘Krizhevsky A. Learning Multiple Layers of Features from Tiny Images. PhD thesis. University of Toronto, 2009.’’

Original

2009

Earlier work this paper cites.

‘‘Нестеров Ю.Е. Введение в выпуклую оптимизацию. М.: МЦНМО, 2010. 262 с.’’

2010

Earlier work this paper cites.

‘‘Duchi J., Hazan E., Singer Y. Adaptive subgradient methods for online learning and stochastic optimization // Journal of Machine Learning Research. 2011. V. 12. № Jul. P. 2121–2159.’’

2011

Earlier work this paper cites.

‘‘Krizhevsky A., Sutskever I., Hinton G. Imagenet classification with deep convolutional neural networks // In Advances in neural information processing systems. 2012. P. 1097–1105.’’

2012

Cited alongside, same era.

‘‘Lan G., Nemirovski A., Shapiro A. Validation analysis of mirror descent stochastic approximation method // Math. Program. 2012. V. 134. № 2. P. 425–458.’’

2012

Cited alongside, same era.

‘‘Devolder O., Glineur F., Nesterov Yu. First-order methods with inexact oracle: the strongly convex case // CORE Discussion Paper 2013/16. 2013. URL: https://www.uclouvain.be/cps/ucl/doc/core/documents/coredp2013_16web.pdf ’’

2013

Cited alongside, same era.

‘‘Devolder O. Exactness, inexactness and stochasticity in first-order methods for large-scale convex optimization. PhD thesis. CORE UCL, 2013.’’

2013

Cited alongside, same era.

‘‘Kingma D.P., Ba J. Adam: a method for stochastic optimization // ICLR, 2015.’’

2015

Later among the works it cites.

‘‘Goodfellow I., Bengio Y., Courville A. Deep learning. MIT press. 2016.’’

2016

Later among the works it cites.

‘‘Гасников А.В., Двуреченский П.Е., Усманова И.Н. О нетривиальности быстрых (ускоренных) рандомизированных методов // Труды МФТИ. 2016. Т. 8. № 2. С. 67–100.’’

2016

Later among the works it cites.

‘‘Гасников А.В. Современные численные методы оптимизации. Метод универсального градиентного спуска // e-print. arXiv:1711.00394. 2018.’’

Original

2018

Later among the works it cites.

‘‘Bach F., Levy K.Y. A universal algorithm for variational inequalities adaptive to smoothness and noise // COLT, 2019.’’

2019

Closest in time.