Analysis and Implementation of an Asynchronous Optimization Algorithm for the Parameter Server

M. V. Solodov, “Incremental gradient algorithms with stepsizes bounded away from zero,” Computational Optimization and Applications , vol. 11, no. 1, pp. 23–35, 1998

1998

Earlier work this paper cites.

D. D. Lewis, Y. Yang, T. G. Rose, and F. Li, “Rcv1: A new benchmark collection for text categorization research,” J. Mach. Learn. Res. , vol. 5, pp. 361–397, Dec. 2004. [Online]. Available: http://dl.acm.org/citation.cfm?id=1005332.1005345

2004

Earlier work this paper cites.

D. Blatt, A. O. Hero, and H. Gauchman, “A convergent incremental gradient method with a constant step size,” SIAM J. Optim. , vol. 18, no. 1, pp. 29–51, jan 2007. [Online]. Available: http://dx.doi.org/10.1137/040615961

2007

Earlier work this paper cites.

J. Ma, L. K. Saul, S. Savage, and G. M. Voelker, “Identifying suspicious urls: An application of large-scale online learning,” in Proceedings of the 26th Annual International Conference on Machine Learning (ICML 2009) , ser. ICML ’09, ACM. Montreal, Quebec: ACM, June 2009, pp. 681–688. [Online]. Available: http://doi.acm.org/10.1145/1553374.1553462

2009

Earlier work this paper cites.

B. Recht, C. Re, S. Wright, and F. Niu, “Hogwild: A lock-free approach to parallelizing stochastic gradient descent,” in Advances in Neural Information Processing Systems 24 , 2011, pp. 693–701

2011

Earlier work this paper cites.

A. Agarwal and J. C. Duchi, “Distributed delayed stochastic optimization,” in Advances in Neural Information Processing Systems , 2011, pp. 873–881

2011

Earlier work this paper cites.

D. P. Bertsekas, “Incremental gradient, subgradient, and proximal methods for convex optimization: A survey,” Optimization for Machine Learning , vol. 2010, pp. 1–38, 2011

2011

Cited alongside, same era.

Y. Nesterov, “Efficiency of coordinate descent methods on huge-scale optimization problems,” SIAM Journal on Optimization , vol. 22, no. 2, pp. 341–362, 2012

2012

Cited alongside, same era.

O. Dekel, R. Gilad-Bachrach, O. Shamir, and L. Xiao, “Optimal distributed online prediction using mini-batches,” Journal of Machine Learning Research , vol. 13, pp. 165–202, 2012

2012

Cited alongside, same era.

M. Li, L. Zhou, Z. Yang, A. Li, F. Xia, D. G. Andersen, and A. Smola, “Parameter server for distributed machine learning,” in Big Learning NIPS Workshop , vol. 1, 2013

2013

Cited alongside, same era.

M. Gurbuzbalaban, A. Ozdaglar, and P. Parrilo, “On the convergence rate of incremental aggregated gradient algorithms,” jun 2015, arXiv: 1506.02081v1 [math.OC]

2015

Later among the works it cites.

J. Mairal, “Incremental majorization-minimization optimization with application to large-scale machine learning,” SIAM Journal on Optimization , vol. 25, no. 2, pp. 829–855, 2015

2015

Later among the works it cites.

H. R. Feyzmahdavian, A. Aytekin, and M. Johansson, “An asynchronous mini-batch algorithm for regularized stochastic optimization,” IEEE Transactions on Automatic Control , vol. PP, no. 99, pp. 1–1, 2016

2016

Closest in time.

N. D. Vanli, M. Gurbuzbalaban, and A. Ozdaglar, “Global convergence rate of proximal incremental aggregated gradient methods,” aug 2016, arXiv: 1608.01713v1 [math.OC]

2016

Closest in time.