Fetching the paper…
Reading the bibliography…
We propose a new integrated method of exploiting model, batch and domain parallelism for the training of deep neural networks (DNNs) on large distributed-memory computers using minibatch stochastic gradient descent (SGD).
A cellular computer to implement the Kalman filter algorithm
Lynn Elliot Cannon · 1969
Earlier work this paper cites.
SUMMA: Scalable universal matrix multiplication algorithm
Robert A Van De Geijn and Jerrell Watts · 1997
Earlier work this paper cites.
Using the BSP cost model to optimise parallel neural network training
RO Rogers and David B Skillicorn · 1998
Earlier work this paper cites.
Optimization of collective communication operations in MPICH
Rajeev Thakur, Rolf Rabenseifner, and William Gropp · 2005
Earlier work this paper cites.
Collective communication: theory, practice, and experience
Ernie Chan, Marcel Heimlich, Avi Purkayastha, and Robert Van De Geijn · 2007
Earlier work this paper cites.
Minimizing communication in numerical linear algebra
Grey Ballard, James Demmel, Olga Holtz, and Oded Schwartz · 2011
Earlier work this paper cites.
Hogwild: A lock-free approach to parallelizing stochastic gradient descent
Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu · 2011
Earlier work this paper cites.
Large scale distributed deep networks
Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, et al · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Deep learning with COTS HPC systems
Adam Coates, Brody Huval, Tao Wang, David Wu, Bryan Catanzaro, and Ng Andrew · 2013
Earlier work this paper cites.
Project adam: Building an efficient and scalable deep learning training system
Trishul M Chilimbi, Yutaka Suzue, Johnson Apacible, and Karthik Kalyanaraman · 2014
Cited alongside, same era.
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, and Trevor Darrell · 2015
Cited alongside, same era.
Faster R-CNN: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun · 2015
Cited alongside, same era.
Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich · 2015
Cited alongside, same era.
Deep learning with elastic averaging SGD
Sixin Zhang, Anna E Choromanska, and Yann LeCun · 2015
Cited alongside, same era.
Distributed deep learning using synchronous stochastic gradient descent
Communication-avoiding parallel sparse-dense matrix-matrix multiplication
Penporn Koanantakool, Ariful Azad, Aydın Buluç, Dmitriy Morozov, Sang-Yun Oh, Leonid Oliker, and Katherine Yelick · 2016
Later among the works it cites.
Parallel matrix multiplication: A systematic journey
Martin D Schatz, Robert A Van de Geijn, and Jack Poulson · 2016
Later among the works it cites.
Bichen Wu, Forrest Iandola, Peter H Jin, and Kurt Keutzer · 2016
Later among the works it cites.
Accurate, large minibatch SGD: Training ImageNet in 1 hour
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He · 2017
Closest in time.
Brain tumor segmentation with deep neural networks
Mohammad Havaei, Axel Davy, David Warde-Farley, Antoine Biard, Aaron Courville, Yoshua Bengio, Chris Pal, Pierre-Marc Jodoin, and Hugo Larochelle · 2017
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Dipankar Das, Sasikanth Avancha, Dheevatsa Mudigere, Karthikeyan Vaidynathan, Srinivas Sridharan, Dhiraj Kalamkar, Bharat Kaul, and Pradeep Dubey · 2016
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
How to scale distributed deep learning?
Peter H Jin, Qiaochu Yuan, Forrest Iandola, and Kurt Keutzer · 2016
Cited alongside, same era.
On large-batch training for deep learning: Generalization gap and sharp minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang · 2016
Cited alongside, same era.
Accurate image super-resolution using very deep convolutional networks
Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee · 2016
Cited alongside, same era.
Closest in time.
SIBIA-GlS: Scalable biophysics-based image analysis for glioma segmentation
A. Mang, S. Tharakan A. Gholami, N. Himthani, S. Subramanian, J. Levitt, M. Azmat, K. Scheufele, M. Mehl, C. Davatzikos, B. Barth, and G. Biros · 2017
Closest in time.
Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud
Bichen Wu, Alvin Wan, Xiangyu Yue, and Kurt Keutzer · 2017
Closest in time.
Scaling SGD batch size to 32k for ImageNet training
Yang You, Igor Gitman, and Boris Ginsburg · 2017
Closest in time.
Yang You, Zhao Zhang, C Hsieh, James Demmel, and Kurt Keutzer · 2017
Closest in time.
Spatially parallel convolutions
Peter Jin, Boris Ginsburg, and Kurt Keutzer · 2018
Closest in time.