Fetching the paper…
Reading the bibliography…
In this work, we present a family of vector quantization schemes \emph{vqSGD} (Vector-Quantized Stochastic Gradient Descent) that provide an asymptotic reduction in the communication cost with convergence guarantees in first-order distributed optimization.
Randomized response: A survey technique for eliminating evasive answer bias
Stanley L Warner · 1965
Earlier work this paper cites.
The theory of error-correcting codes
Florence Jessie MacWilliams and Neil James Alexander Sloane · 1977
Earlier work this paper cites.
Simple approximations of the error function q(x) for communications applications
P Borjesson and C-E Sundberg · 1979
Earlier work this paper cites.
Covering codes
Gérard Cohen, Iiro Honkala, Simon Litsyn, and Antoine Lobstein · 1997
Earlier work this paper cites.
Hadamard matrices, orthogonal designs and construction algorithms
Stelios Georgiou, Christos Koukouvinos, and Jennifer Seberry · 2003
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
Boosting and differential privacy
Cynthia Dwork, Guy N Rothblum, and Salil Vadhan · 2010
Earlier work this paper cites.
MNIST handwritten digit database
Yann LeCun and Corinna Cortes · 2010
Earlier work this paper cites.
Trading accuracy for sparsity in optimization problems with sparsity constraints
Shai Shalev-Shwartz, Nathan Srebro, and Tong Zhang · 2010
Earlier work this paper cites.
Libsvm: A library for support vector machines
Chih-Chung Chang and Chih-Jen Lin · 2011
Earlier work this paper cites.
Information-theoretic lower bounds for distributed statistical estimation with communication constraints
Yuchen Zhang, John Duchi, Michael I Jordan, and Martin J Wainwright · 2013
Earlier work this paper cites.
The algorithmic foundations of differential privacy
Cynthia Dwork, Aaron Roth, et al · 2014
Earlier work this paper cites.
Rappor: Randomized aggregatable privacy-preserving ordinal response
Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova · 2014
Earlier work this paper cites.
1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns
Frank Seide, Hao Fu, Jasha Droppo, Gang Li, and Dong Yu · 2014
Cited alongside, same era.
Understanding machine learning: From theory to algorithms
Shai Shalev-Shwartz and Shai Ben-David · 2014
Cited alongside, same era.
Privacy-preserving deep learning
Reza Shokri and Vitaly Shmatikov · 2015
Cited alongside, same era.
Scalable distributed DNN training using commodity GPU cloud computing
Nikko Strom · 2015
Cited alongside, same era.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Cited alongside, same era.
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith · 2016
Cited alongside, same era.
The convergence of sparsified gradient methods
Dan Alistarh, Torsten Hoefler, Mikael Johansson, Nikola Konstantinov, Sarit Khirirat, and Cédric Renggli · 2018
Later among the works it cites.
signSGD: Compressed optimization for non-convex problems
Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, and Animashree Anandkumar · 2018
Later among the works it cites.
Sparsified SGD with memory
Sebastian U Stich, Jean-Baptiste Cordonnier, and Martin Jaggi · 2018
Later among the works it cites.
Atomo: Communication-efficient learning via atomic sparsification
Hongyi Wang, Scott Sievert, Shengchao Liu, Zachary Charles, Dimitris Papailiopoulos, and Stephen Wright · 2018
Later among the works it cites.
Distributed learning with sublinear communication
Jayadev Acharya, Chris De Sa, Dylan Foster, and Karthik Sridharan · 2019
Closest in time.
Hadamard response: Estimating distributions privately, efficiently, and with little communication
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Federated learning: Strategies for improving communication efficiency
Jakub Konečnỳ, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon · 2016
Cited alongside, same era.
QSGD: Communication-efficient SGD via gradient quantization and encoding
Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic · 2017
Cited alongside, same era.
Convex optimization: Algorithms and complexity
Sébastien Bubeck · 2017
Cited alongside, same era.
Distributed mean estimation with limited communication
Ananda Theertha Suresh, Felix X Yu, Sanjiv Kumar, and H Brendan McMahan · 2017
Cited alongside, same era.
Terngrad: Ternary gradients to reduce communication in distributed deep learning
Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li · 2017
Cited alongside, same era.
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017
Han Xiao, Kashif Rasul, and Roland Vollgraf · 2017
Cited alongside, same era.
Jayadev Acharya, Ziteng Sun, and Huanyu Zhang · 2019
Closest in time.
Stochastic Distributed Learning with Gradient Quantization and Variance Reduction
Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Sebastian Stich, and Peter Richtárik · 2019
Closest in time.
Communication-efficient distributed SGD with sketching
Nikita Ivkin, Daniel Rothchild, Enayat Ullah, Ion Stoica, Raman Arora, et al · 2019
Closest in time.
Fitting ReLus via SGD and quantized SGD
Seyed Mohammadreza Mousavi Kalan, Mahdi Soltanolkotabi, and A Salman Avestimehr · 2019
Closest in time.
Error feedback fixes signSGD and other gradient compression schemes
Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian Stich, and Martin Jaggi · 2019
Closest in time.
Decentralized stochastic optimization and gossip algorithms with compressed communication
Anastasiia Koloskova, Sebastian Urban Stich, and Martin Jaggi · 2019
Closest in time.
Ratq: A universal fixed-length quantizer for stochastic optimization
Prathamesh Mayekar and Himanshu Tyagi · 2019
Closest in time.
High-dimensional statistics: A non-asymptotic viewpoint
Martin J Wainwright · 2019
Closest in time.