Fetching the paper…
Reading the bibliography…
Decentralized optimization methods enable on-device training of machine learning models without a central coordinator.
Television by pulse code modulation
W. M. Goodall · 1951
Earlier work this paper cites.
Picture coding using pseudo-random noise
L. Roberts · 1962
Earlier work this paper cites.
On the solution of a class of non linear dirichlet problems by a penalty-duality method and finite elements of order one
R. Glowinski and A. Marrocco · 1975
Earlier work this paper cites.
A dual algorithm for the solution of nonlinear variational problems via finite element approximation
Daniel Gabay and Bertrand Mercier · 1976
Earlier work this paper cites.
Constrained Optimization and Lagrange Multiplier Methods
Dimitri P. Bertsekas · 1982
Earlier work this paper cites.
Problems in decentralized decision making and computation
John N. Tsitsiklis · 1984
Earlier work this paper cites.
Convergence rates in forward–backward splitting
George H-G. Chen and R. T. Rockafellar · 1997
Earlier work this paper cites.
Fast training of support vector machines using sequential minimal optimization
John C. Platt · 1998
Earlier work this paper cites.
A unified theory of decentralized SGD with changing topology and local updates
Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, and Sebastian U. Stich · 2003
Earlier work this paper cites.
A scheme for robust distributed sensor fusion based on average consensus
L. Xiao, S. Boyd, and S. Lall · 2005
Earlier work this paper cites.
Distributed subgradient methods and quantization effects
Angelia Nedić, Alex Olshevsky, Asuman Ozdaglar, and John N. Tsitsiklis · 2008
Earlier work this paper cites.
Distributed subgradient methods for multi-agent optimization
A. Nedić and A. Ozdaglar · 2009
Earlier work this paper cites.
Distributed optimization and statistical learning via the alternating direction method of multipliers
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein · 2011
Earlier work this paper cites.
Large scale distributed deep networks
Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc D’aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, Quoc V. Le, and Andrew Y. Ng · 2012
Earlier work this paper cites.
Dual averaging for distributed optimization: Convergence analysis and network scaling
J. C. Duchi, A. Agarwal, and M. J. Wainwright · 2012
Earlier work this paper cites.
Distributed alternating direction method of multipliers
E. Wei and A. Ozdaglar · 2012
Earlier work this paper cites.
Distributed dual averaging method for multi-agent optimization with quantized communication
Deming Yuan, Shengyuan Xu, Huanyu Zhao, and Lina Rong · 2012
Earlier work this paper cites.
Asynchronous distributed optimization using a randomized alternating direction method of multipliers
Franck Iutzeler, Pascal Bianchi, Philippe Ciblat, and Walid Hachem · 2013
Earlier work this paper cites.
Accelerating stochastic gradient descent using predictive variance reduction
Rie Johnson and Tong Zhang · 2013
Earlier work this paper cites.
SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives
Aaron Defazio, Francis Bach, and Simon Lacoste-Julien · 2014
Earlier work this paper cites.
Adaptation, learning, and optimization over networks
Ali Sayed · 2014
Earlier work this paper cites.
1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs
Frank Seide, Hao Fu, Jasha Droppo, Gang Li, and Dong Yu · 2014
Earlier work this paper cites.
On the linear convergence of the ADMM in decentralized consensus optimization
W. Shi, Q. Ling, K. Yuan, G. Wu, and W. Yin · 2014
Cited alongside, same era.
A universal catalyst for first-order optimization
Hongzhou Lin, Julien Mairal, and Zaid Harchaoui · 2015
Cited alongside, same era.
Multi-agent mirror descent for decentralized stochastic optimization
M. Rabbat · 2015
Cited alongside, same era.
EXTRA: An exact first-order algorithm for decentralized consensus optimization
Wei Shi, Qing Ling, Gang Wu, and Wotao Yin · 2015
Cited alongside, same era.
Scalable distributed dnn training using commodity gpu cloud computing
Nikko Strom · 2015
Cited alongside, same era.
Quantized decentralized consensus optimization
Amirhossein Reisizadeh, Aryan Mokhtari, S. Hamed Hassani, and Ramtin Pedarsani · 2018
Later among the works it cites.
Sparsified SGD with memory
Sebastian U Stich, Jean-Baptiste Cordonnier, and Martin Jaggi · 2018
Later among the works it cites.
Communication compression for decentralized training
Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, and Ji Liu · 2018
Later among the works it cites.
A dual approach for optimal algorithms in distributed optimization over networks
César A Uribe, Soomin Lee, and Alexander Gasnikov · 2018
Later among the works it cites.
Jianyu Wang and Gauri Joshi · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
H. Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Agüera y Arcas · 2016
Cited alongside, same era.
DSA: Decentralized double stochastic averaging gradient algorithm
Aryan Mokhtari and Alejandro Ribeiro · 2016
Cited alongside, same era.
Stochastic variance reduction methods for saddle-point problems
Balamurugan Palaniappan and Francis Bach · 2016
Cited alongside, same era.
Harnessing smoothness to accelerate distributed optimization
G. Qu and N. Li · 2016
Cited alongside, same era.
CNTK: Microsoft’s open-source deep-learning toolkit
Frank Seide and Amit Agarwal · 2016
Cited alongside, same era.
On the convergence of decentralized gradient descent
Kun Yuan, Qing Ling, and Wotao Yin · 2016
Cited alongside, same era.
QSGD: Communication-efficient SGD via gradient quantization and encoding
Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic · 2017
Cited alongside, same era.
Gradient sparsification for communication-efficient distributed optimization
Jianqiao Wangni, Jialei Wang, Ji Liu, and Tong Zhang · 2018
Later among the works it cites.
A linearly convergent proximal gradient algorithm for decentralized optimization
Sulaiman Alghunaim, Kun Yuan, and Ali H Sayed · 2019
Later among the works it cites.
Stochastic distributed learning with gradient quantization and variance reduction
Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Peter Richtárik, and Sebastian U. Stich · 2019
Later among the works it cites.
Advances and open problems in federated learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D’Oliveira, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaïd Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konecný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao · 2019
Later among the works it cites.
Error feedback fixes SignSGD and other gradient compression schemes
Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian U. Stich, and Martin Jaggi · 2019
Later among the works it cites.
Decentralized stochastic optimization and gossip algorithms with compressed communication
Anastasia Koloskova, Sebastian U. Stich, and Martin Jaggi · 2019
Later among the works it cites.
An exact quantized decentralized gradient descent algorithm
A. Reisizadeh, A. Mokhtari, H. Hassani, and R. Pedarsani · 2019
Later among the works it cites.
Sebastian U. Stich and Sai P. Karimireddy · 2019
Later among the works it cites.
Deepsqueeze: Decentralization meets error-compensated compression
Hanlin Tang, Xiangru Lian, Shuang Qiu, Lei Yuan, Ce Zhang, Tong Zhang, and Ji Liu · 2019
Later among the works it cites.
Variance-reduced decentralized stochastic optimization with accelerated convergence
Ran Xin, Usman A. Khan, and Soummya Kar · 2019
Later among the works it cites.
Linear convergence of primal-dual gradient methods and their performance in distributed optimization
Sulaiman A. Alghunaim and Ali H. Sayed · 2020
Closest in time.
An optimal algorithm for decentralized finite sum optimization
Hadrien Hendrikx, Francis Bach, and Laurent Massoulie · 2020
Closest in time.
Linear convergent decentralized optimization with compression
Xiaorui Liu, Yao Li, Rongrong Wang, Jiliang Tang, and Ming Yan · 2020
Closest in time.
Distributed gradient methods for convex machine learning problems in networks: Distributed optimization
Angelia Nedić · 2020
Closest in time.
Distributed stochastic gradient tracking methods
S. Pu and A. Nedić · 2020
Closest in time.
Decentralized stochastic optimization and machine learning: A unified variance-reduction framework for robust performance and fast convergence
R. Xin, S. Kar, and U. A. Khan · 2020
Closest in time.