Fetching the paper…
Reading the bibliography…
Training time on large datasets for deep neural networks is the principal workflow bottleneck in a number of important applications of deep learning, such as object classification and detection in automatic driver assistance systems (ADAS).
MPI: A Standard Message Passing Interface
David W. Walker and Jack J. Dongarra · 1996
Earlier work this paper cites.
Gossip-Based Computation of Aggregate Information
David Kempe, Alin Dobra, and Johannes Gehrke · 2003
Earlier work this paper cites.
Optimization of Collective Communication Operations in MPICH
Rajeev Thakur, Rolf Rabenseifner, and William Gropp · 2005
Earlier work this paper cites.
Randomized Gossip Algorithms
Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah · 2006
Earlier work this paper cites.
Asynchronous Gossip Algorithms for Stochastic Optimization
S. Sundhar Ram, A. Nedić, and V. V. Veeravalli · 2009
Earlier work this paper cites.
Asynchronous Stochastic Convex Optimization for Random Networks: Error Bounds
Behrouz Touri, A. Nedić, and S. Sundhar Ram · 2010
Earlier work this paper cites.
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu · 2011
Earlier work this paper cites.
Large Scale Distributed Deep Networks
Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V. Le, and Andrew Y. Ng · 2012
Earlier work this paper cites.
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton · 2012
Earlier work this paper cites.
Lecture 6.5-RMSProp: Divide the gradient by a running average of its recent magnitude
Tijmen Tieleman and Geoffrey Hinton · 2012
Cited alongside, same era.
More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server
Qirong Ho, James Cipar, Henggang Cui, Seunghak Lee, Jin Kyu Kim, Phillip B. Gibbons, Garth A. Gibson, Greg Ganger, and Eric P. Xing · 2013
Cited alongside, same era.
Introductory Lectures on Convex Optimization: A Basic Course , volume 87
Yurii Nesterov · 2013
Cited alongside, same era.
Project Adam: Building an Efficient and Scalable Deep Learning Training System
Trishul Chilimbi, Yutaka Suzue, Johnson Apacible, and Karthik Kalyanaraman · 2014
Cited alongside, same era.
Communication Efficient Distributed Machine Learning with the Parameter Server
Mu Li, David G. Andersen, Alex J. Smola, and Kai Yu · 2014
Cited alongside, same era.
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Later among the works it cites.
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Fei-Fei Li · 2015
Later among the works it cites.
Deep learning with Elastic Averaging SGD
Sixin Zhang, Anna E. Choromanska, and Yann LeCun · 2015
Later among the works it cites.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe and Christian Szegedy · 2015
Later among the works it cites.
Adam: A Method for Stochastic Optimization
Diederik Kingma and Jimmy Ba · 2015
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhammer · 2014
Cited alongside, same era.
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan and Andrew Zisserman · 2014
Cited alongside, same era.
Deep Image: Scaling up Image Recognition
Ren Wu, Shengen Yan, Yi Shan, Qingqing Dang, and Gang Sun · 2015
Cited alongside, same era.
FireCaffe: near-linear acceleration of deep neural network training on compute clusters
Forrest N. Iandola, Khalid Ashraf, Matthew W. Moskewicz, and Kurt Keutzer · 2015
Cited alongside, same era.
Dipankar Das, Sasikanth Avancha, Dheevatsa Mudigere, Karthikeyan Vaidynathan, Srinivas Sridharan, Dhiraj Kalamkar, Bharat Kaul, and Pradeep Dubey · 2016
Closest in time.
Revisiting Distributed Synchronous SGD
Jianmin Chen, Rajat Monga, Samy Bengio, and Rafal Jozefowicz · 2016
Closest in time.
Asynchronous Methods for Deep Reinforcement Learning
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy P Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu · 2016
Closest in time.