Understand
We consider the problem of estimating the arithmetic average of a finite collection of real vectors stored in a distributed fashion across several compute nodes subject to a communication budget constraint.
- Our analysis does not rely on any statistical assumptions about the source of the vectors.
- This problem arises as a subproblem in many applications, including reduce-all operations within algorithms for distributed and federated optimization and learning.
- We propose a flexible family of randomized algorithms exploring the trade-off between expected communication cost and estimation error.
Built on
Communication-efficient algorithms for statistical optimization
Yuchen Zhang, Martin J. Wainwright, and John C. Duchi · 2012
Earlier work this paper cites.
Information-theoretic lower bounds for distributed statistical estimation with communication constraints
Yuchen Zhang, John Duchi, Michael I. Jordan, and Martin J. Wainwright · 2013
Earlier work this paper cites.
On communication cost of distributed statistical estimation and dimensionality
Ankit Garg, Tengyu Ma, and Huy L. Nguyen · 2014
Earlier work this paper cites.
Mark Braverman, Ankit Garg, Tengyu Ma, Huy L. Nguyen, and David P. Woodruff · 2015
Earlier work this paper cites.
Distributed optimization with arbitrary local solvers
Chenxin Ma, Jakub Konečný, Martin Jaggi, Virginia Smith, Michael I. Jordan, Peter Richtárik, and Martin Takáč · 2015
Earlier work this paper cites.
Similar
Adding vs. averaging in distributed primal-dual optimization
Chenxin Ma, Virginia Smith, Martin Jaggi, Michael I. Jordan, Peter Richtárik, and Martin Takáč · 2015
Cited alongside, same era.
Federated optimization: distributed machine learning for on-device intelligence
Jakub Konečný, H. Brendan McMahan, Daniel Ramage, and Peter Richtárik · 2016
Cited alongside, same era.
Federated learning: Strategies for improving communication efficiency
Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon · 2016
Cited alongside, same era.
Federated learning of deep networks using model averaging
H. Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Aguera y Arcas · 2016
Cited alongside, same era.
Aide: Fast and communication efficient distributed optimization
Sashank J. Reddi, Jakub Konečný, Peter Richtárik, Barnabás Póczós, and Alex Smola · 2016
Cited alongside, same era.
Then
Distributed coordinate descent method for learning with big data
Peter Richtárik and Martin Takáč · 2016
Closest in time.
Distributed mean estimation with limited communication
Ananda Theertha Suresh, Felix X. Yu, H. Brendan McMahan, and Sanjiv Kumar · 2016
Closest in time.
Variable-length quantity, 2016
Wikipedia · 2016
Closest in time.
Felix X. Yu, Ananda Theertha Suresh, Krzysztof Choromanski, Daniel Holtmann-Rice, and Sanjiv Kumar · 2016
Closest in time.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…