Fetching the paper…

Reading the bibliography…

2018

Distributed learning with compressed gradients

Khirirat, Sarit, Feyzmahdavian, Hamid Reza, Johansson, Mikael

Original
PDF
Post on X

Understand

Asynchronous computation and gradient compression have emerged as two key techniques for achieving scalability in distributed optimization for large-scale machine learning.

This paper presents a unified analysis framework for distributed gradient methods operating with staled and compressed gradients.
Non-asymptotic bounds on convergence rates and information exchange are derived for several optimization algorithms.
These bounds give explicit expressions for step-sizes and characterize how the amount of asynchrony and the compression accuracy affect iteration and communication complexity guarantees.

Distributed learning with compressed gradients · Around

Built on

B. T. Polyak,
1987
Earlier work this paper cites.
J. N. Tsitsiklis and Z.-Q. Luo,
1987
Earlier work this paper cites.
D. Blatt, A. O. Hero, and H. Gauchman,
2007
Earlier work this paper cites.
Y. Nesterov,
2013
Earlier work this paper cites.
P. Tseng and S. Yun,
2014
Earlier work this paper cites.
F. Seide, H. Fu, J. Droppo, G. Li, and D. Yu,
2014
Earlier work this paper cites.
M. Li, D. G. Andersen, A. J. Smola, and K. Yu,
2014
Earlier work this paper cites.
R. Zhang and J. Kwok,
2014
Earlier work this paper cites.

Similar

D. Needell, R. Ward, and N. Srebro,
2014
Cited alongside, same era.
A. Aytekin, H. R. Feyzmahdavian, and M. Johansson,
2014
Cited alongside, same era.
C.M. De Sa, C. Zhang and C. Ré,
2015
Cited alongside, same era.
D. Alistarh, D. Grubic, J. Li, R. Tomioka and M. Vojnovic,
Original
2016
Cited alongside, same era.
A. Aytekin, H.R. Feyzmahdavian and M. Johansson,
Original
2016
Cited alongside, same era.
J. Chen, X. Pan, R. Monga, S. Bengio, and R. Jozefowicz,
Original
2016
Cited alongside, same era.

Then

M. Gurbuzbalaban, A. Ozgaglar and P. Parrilo,
2017
Later among the works it cites.
S. Khirirat, H. R. Feyzmahdavian, and M. Johansson,
2017
Later among the works it cites.
S. Magnússon, C. Enyioha, N. Li, C. Fischione, and V. Tarokh,
2017
Later among the works it cites.
W. Wen, C. Xu, F. Yan, C. Wu, Y. Wang, Y. Chen, and H. Li,
2017
Later among the works it cites.
S. Khirirat, H. R. Feyzmahdavian, and M. Johansson,
Original
2017
Later among the works it cites.
S. Khirirat, H.R. Feyzmahdavian and M. Johansson,
Original
2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

O. Shamir,

2016

Cited alongside, same era.

J. Wangni, J. Wang, J. Liu and T. Zhang,

2017

Cited alongside, same era.

L. M. Nguyen, P. H. Nguyen, M. van Dijk, P. Richtárik, K. Scheinberg, and M. Takáč,

2018

Closest in time.