Improving Distributed Gradient Descent Using Reed-Solomon Codes

Polynomial codes over certain finite fields

Irving Reed and Gus Solomon · 1960

Earlier work this paper cites.

Solution of vandermonde systems of equations

Ake Björck and Victor Pereyra · 1970

Earlier work this paper cites.

Estimators based on order statistics from a pareto distribution

Kerstin Vännman · 1976

Earlier work this paper cites.

Load-balancing heuristics and process behavior

Will Leland and Teunis J Ott · 1986

Earlier work this paper cites.

Exploiting process lifetime distributions for dynamic load balancing

Mor Harchol-Balter and Allen B Downey · 1997

Earlier work this paper cites.

The mnist database of handwritten digits, 1998

Yann LeCun, Corinna Cortes, and Christopher JC Burges · 1998

Earlier work this paper cites.

The effect of heavy-tailed job size distributions on computer system design

Mor Harchol-Balter · 1999

Earlier work this paper cites.

Parallelized stochastic gradient descent

Original

Martin Zinkevich, Markus Weimer, Lihong Li, and Alex J Smola · 2010

Cited alongside, same era.

Distributed optimization and statistical learning via the alternating direction method of multipliers

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein · 2011

Cited alongside, same era.

Large-scale matrix factorization with distributed stochastic gradient descent

Rainer Gemulla, Erik Nijkamp, Peter J Haas, and Yannis Sismanis · 2011

Cited alongside, same era.

Hogwild: A lock-free approach to parallelizing stochastic gradient descent

Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu · 2011

Cited alongside, same era.

Introductory lectures on convex optimization: A basic course

Yurii Nesterov · 2013

Cited alongside, same era.

Balanced Reed-Solomon codes

W Halbawi, Z Liu, and B Hassibi · 2016

Later among the works it cites.

Balanced Reed-Solomon codes for all parameters

W Halbawi, Z Liu, and B Hassibi · 2016

Later among the works it cites.

Speeding up distributed machine learning using codes

Kangwook Lee, Maximilian Lam, Ramtin Pedarsani, Dimitris Papailiopoulos, and Kannan Ramchandran · 2016

Later among the works it cites.

Fundamental tradeoff between computation and communication in distributed computing

Songze Li, Mohammad Ali Maddah-Ali, and A Salman Avestimehr · 2016

Later among the works it cites.

Gradient Coding

Rashish Tandon, Qi Lei, Alexandros G. Dimakis, and Nikos Karampatziakis · 2016

Later among the works it cites.

Revisiting distributed synchronous sgd

Original

Xinghao Pan, Jianmin Chen, Rajat Monga, Samy Bengio, and Rafal Jozefowicz · 2017