Fetching the paper…
Reading the bibliography…
We propose dynamic sampled stochastic approximation (SA) methods for stochastic optimization with a heavy-tailed distribution (with finite 2nd moment).
H. Robbins and S. Monro, A Stochastic Approximation Method , The Annals of Mathematical Statistics, 22 (1951), pp. 400-407
1951
Earlier work this paper cites.
K. L. Chung, On a stochastic approximation method , Ann. Math. Statist., 25 (1954), pp. 463-483
1954
Earlier work this paper cites.
A. Dvoretzky, On Stochastic Approximation , in: Proc. Third Berkeley Symp. on Math. Statist. and Prob., Vol. 1, Univ. of Calif. Press, 1956, pp. 39-55
1956
Earlier work this paper cites.
A.S. Nemirovski and D.B. judin, On Cezari’s convergence of the steepest descent method for approximating saddle point of convex-concave functions , Soviet Mathematics-Doklady, 19 (1978)
1978
Earlier work this paper cites.
Y. Nesterov, A method for unconstrained convex minimization problem with the rate of convergence O ( 1 / k 2 ) O(1/k^{2}) , Soviet Mathematics Doklady, 27 (1983), pp. 372-376
1983
Earlier work this paper cites.
D. Ruppert, Efficient estimations from a slowly convergent Robbins-Monro process , tech. report, Cornell University Operations Research and Industrial Engineering,(1988), preprint at https://ecommons.cornell.edu/handle/1813/8664
1988
Earlier work this paper cites.
B.T. Polyak, New Method of Stochastic Approximation Type , Automation and Remote Control, 51 (1991), pp. 937-946
1991
Earlier work this paper cites.
B.T. Polyak and A.B. Juditsky, Acceleration of Stochastic Approximation by Averaging , SIAM Journal on Control and Optimization, 30 (1992), pp. 838-855
1992
Earlier work this paper cites.
A.B. Juditsky, A.V. Nazin, A.B. Tsybakov and N. Vayatis, Recursive aggregation of estimators via the mirror descent algorithm with averaging , Probl. Inf. Transm. 41 (2005), Issue 4, pp. 368-384
2005
Earlier work this paper cites.
H. Jiang and H. Xu, Stochastic approximation approaches to the stochastic variational inequality problem , IEEE Transactions on Automatic Control, 53 (2008), Issue 6, pp. 1462-1475
2008
Earlier work this paper cites.
A. Juditsky, P. Rigollet and A.B. Tsybakov, Learning by mirror averaging , Ann. Stat. 36 (2008), No.5, pp. 2183-2206
2008
Earlier work this paper cites.
Y. Nesterov and Vial (2008), Confidence level solutions for stochastic programming , Vol. 44 (2008), Issue 6, pp. 1559-1568
2008
Earlier work this paper cites.
P. Tseng, On Accelerated Proximal Gradient Methods for Convex-Concave Optimization , manuscript, University of Washington, Seattle, 2008
2008
Earlier work this paper cites.
A. Beck and M. Teboulle, A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , SIAM J. Imaging Sci., Vol. 2 (2009), No.1, pp. 183-202
2009
Earlier work this paper cites.
J. Duchi and Y. Singer, Efficient Online and Batch Learning Using Forward Backward Splitting , Journal of Machine Learning Research, 10 (2009), pp. 2899-2934
2009
Earlier work this paper cites.
C. Hu, J. T. Kwok, and W. Pan, Accelerated gradient methods for stochastic optimization and online learning , in Advances in Neural Information Processing Systems (NIPS), (2009)
2009
Earlier work this paper cites.
A. Nemirovski, A. Juditsky, G. Lan and A. Shapiro, Robust stochastic approximation approach to stochastic programming , SIAM J. Optim., Vol. 19 (2009), No. 4, pp. 1574-1609
2009
Earlier work this paper cites.
Y. Nesterov, Primal-dual subgradient methods for convex problems , Mathematical Programming Ser. B, 120 (2009), Issue 1, pp. 221-259
2009
Earlier work this paper cites.
A. Shapiro, D. Dentcheva and A. Ruszczyński, Lectures on Stochastic Programming: Modeling and Theory , MOS-SIAM Ser. Optim., SIAM, Philadelphia, 2009
2009
Earlier work this paper cites.
L. Xiao, Dual averaging methods for regularized stochastic learning and online optimization , Journal of Machine Learning Research, Vol. 9 (2010), pp. 2543-2596
2010
Cited alongside, same era.
F. Bach and E. Moulines, Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning , conference paper, Advances in Neural Information Processing Systems (NIPS), (2011)
2011
Cited alongside, same era.
A. Juditsky, A. Nemirovski and C. Tauvel, Solving variational inequalities with stochastic mirror-prox algorithm , Stochastic Systems, Vol. 1 (2011), No.1, pp. 17-58
2011
Cited alongside, same era.
A. Agarwal, P. Barlett, P. Ravikumar and M.J. Wainwright, Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization , IEEE Transactions on Information Theory, Vol. 58 (2012), Issue 5, pp. 3235-3249
2012
Cited alongside, same era.
R. Frostig, R. Ge, S.M. Kakade and A. Sidford, Competing with the Empirical Risk Minimizer in a Single Pass , in: COLT 2015 Proceedings, (2015)
2015
Later among the works it cites.
M.C. Fu (Editor), Handbook of simulation optimization , Springer, New York, 2015
2015
Later among the works it cites.
2015
Later among the works it cites.
H. Lin, J. Mairal, and Z. Harchaoui, A Universal Catalyst for First-Order Optimization , in Advances in Neural Information Processing Systems (NIPS), (2015)
2015
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
R.H. Byrd, G.M. Chin, J. Nocedal and Y. Wu, Sample Size selection in Optimization Methods for Machine Learning , Mathematical Programming ser. B, Vol. 134 (2012), Issue 1, pp. 127-155
2012
Cited alongside, same era.
M. Friedlander and M. Schmidt, Hybrid deterministic-stochastic methods for data fitting , SIAM J. Sci. Comput., Vol. 34 (2012), Issue 3, pp. 1380-1405
2012
Cited alongside, same era.
S. Ghadimi and G. Lan, Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, I: A generic algorithmic framework , SIAM J. Optim., Vol. 22, (2012), No.4, pp. 1469-1492
2012
Cited alongside, same era.
G. Lan, An optimal method for stochastic composite optimization , Mathematical Programming ser. A, Vol. 133 (2012), Issue 1, pp 365-397
2012
Cited alongside, same era.
N. Le Roux, M. Schmidt, and F. R. Bach, A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets , in Advances in Neural Information Processing Systems 25 (NIPS), (2012)
2012
Cited alongside, same era.
S. Lee and S. Wright, Manifold identification in dual averaging for regularized stochastic online learning , Journal of Machine Learning Research, Vol. 13 (2012), pp. 1705-1744
2012
Cited alongside, same era.
S. Sra, S. Nowozin and S.J. Wright (Editors), Optimization for Machine Learning , The MIT Press, Cambridge, Massachusetts, 2012
2012
Cited alongside, same era.
R. Johnson and T. Zhang, Accelerating stochastic gradient descent using predictive variance reduction , in Advances in Neural Information Processing Systems (NIPS), (2013)
2013
Cited alongside, same era.
2016
Later among the works it cites.
P. Balamurugan and F. Bach, Stochastic Variance Reduction Methods for Saddle-Point Problems , conference paper, Advances in Neural Information Processing Systems (NIPS), (2016)
2016
Later among the works it cites.
2016
Later among the works it cites.
2016
Later among the works it cites.
S. Ghadimi and G. Lan, Accelerated gradient methods for nonconvex nonlinear and stochastic programming , Mathematical Programming ser. A, Vol. 156 (2016), Issue 1, pp. 59-99
2016
Later among the works it cites.
2016
Later among the works it cites.
2016
Later among the works it cites.
D. Needell, N. Srebro and R. Ward Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm , Mathematical Programming ser. A, Vol. 155 (2016), Issue 1, pp. 549-573
2016
Later among the works it cites.
B.E. Woodworth and N. Srebro, Tight Complexity Bounds for Optimizing Composite Objectives , in: Advances in Neural Information Processing Systems 29 (NIPS), 2016
2016
Later among the works it cites.
2017
Closest in time.
A. Iusem, A. Jofré, R.I. Oliveira and P. Thompson, Extragradient methods with variance reduction for stochastic variational inequalities , SIAM J. Optim., Vol.27 (2017), No.2, pp. 686-724
2017
Closest in time.
2017
Closest in time.
M. Schmidt, N. Le Roux, and F. Bach, Minimizing finite sums with the stochastic average gradient , Mathematical Programming ser. A, Vol. 162 (2017), Issue 1, pp. 83-112
2017
Closest in time.
L. Xiao and T. Zhang, A Proximal Stochastic Gradient Method with Progressive Variance Reduction , SIAM Journal on Optimization, Vol. 24 (2014), No.4, pp. 2057-2075
2075
Closest in time.
S. Ghadimi and G. Lan, Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, II: shrinking procedures and optimal algorithms , SIAM J. Optim., Vol. 23 (2013), No.4, pp. 2061-2089
2089
Closest in time.