Fetching the paper…
Reading the bibliography…
Scaling CNN training is necessary to keep up with growing datasets and reduce training time.
P. Fraigniaud and E. Lazard, “Methods and problems of communication in usual networks,” Discrete Applied Mathematics , vol. 53, 1994
1994
Earlier work this paper cites.
R. Thakur, R. Rabenseifner, and W. Gropp, “Optimization of collective communication operations in MPICH,” The International Journal of High Performance Computing Applications , vol. 19, no. 1, 2005
2005
Earlier work this paper cites.
R. Raina, A. Madhavan, and A. Y. Ng, “Large-scale deep unsupervised learning using graphics processors,” in ICML , 2009
2009
Earlier work this paper cites.
S. Oh et al. , “A large-scale benchmark dataset for event recognition in surveillance video,” in CVPR , 2011
2011
Earlier work this paper cites.
N. Maruyama et al. , “Physis: An implicitly parallel programming model for stencil computations on large-scale GPU-accelerated supercomputers,” in SC , 2011
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in NIPS , 2012
2012
Earlier work this paper cites.
J. Dean et al. , “Large scale distributed deep networks,” in NIPS , 2012
2012
Earlier work this paper cites.
J. Poulson et al. , “Elemental: A new framework for distributed memory dense matrix computations,” ACM TOMS , vol. 39, no. 2, 2013
2013
Earlier work this paper cites.
A. Coates et al. , “Deep learning with COTS HPC systems,” in ICML , 2013
2013
Earlier work this paper cites.
2014
Earlier work this paper cites.
T. M. Chilimbi et al. , “Project Adam: Building an efficient and scalable deep learning training system.” in OSDI , vol. 14, 2014
2014
Earlier work this paper cites.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521, no. 7553, 2015
2015
Earlier work this paper cites.
O. Russakovsky et al. , “ImageNet Large Scale Visual Recognition Challenge,” IJCV , vol. 115, no. 3, 2015
2015
Earlier work this paper cites.
B. Van Essen et al. , “LBANN: Livermore big artificial neural network HPC toolkit,” in MLHPC , 2015
2015
Earlier work this paper cites.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in ICML , 2015
2015
Earlier work this paper cites.
M. D. Schatz, “Distributed tensor computations: formalizing distributions, redistributions, and algorithm derivation,” Ph.D. dissertation, University of Texas at Austin, 2015
2015
Earlier work this paper cites.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in CVPR , 2015
2015
Cited alongside, same era.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in ICLR , 2015
2015
Cited alongside, same era.
F. Yan et al. , “Performance modeling and scalability optimization of distributed deep learning systems,” in SIGKDD , 2015
2015
Cited alongside, same era.
2016
Cited alongside, same era.
T. N. Mundhenk et al. , “A large contextual dataset for classification, detection and counting of cars with deep learning,” in ECCV , 2016
2016
Cited alongside, same era.
C. Meng et al. , “Training deeper models by GPU memory optimization on TensorFlow,” in ML Systems Workshop @ NIPS , 2017
2017
Later among the works it cites.
2018
Later among the works it cites.
Top 500, “June 2018 TOP500,” https://www.top500.org/lists/2018/06/ , 2018
2018
Later among the works it cites.
H. Chen et al. , “The rise of deep learning in drug discovery,” Drug Discovery Today , 2018
2018
Later among the works it cites.
2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
K. He et al. , “Deep residual learning for image recognition,” in CVPR , 2016
2016
Cited alongside, same era.
M. D. Schatz, R. A. Van de Geijn, and J. Poulson, “Parallel matrix multiplication: A systematic journey,” SIAM Journal on Scientific Computing , vol. 38, no. 6, 2016
2016
Cited alongside, same era.
2016
Cited alongside, same era.
M. Rhu et al. , “vDNN: Virtualized deep neural networks for scalable, memory-efficient neural network design,” in MICRO , 2016
2016
Cited alongside, same era.
C. Sun et al. , “Revisiting unreasonable effectiveness of data in deep learning era,” in ICCV , 2017
2017
Cited alongside, same era.
G. Litjens et al. , “A survey on deep learning in medical image analysis,” Medical Image Analysis , vol. 42, 2017
2017
Cited alongside, same era.
N. S. Keskar et al. , “On large-batch training for deep learning: Generalization gap and sharp minima,” in ICLR , 2017
2017
Cited alongside, same era.
Y. You et al. , “ImageNet training in minutes,” in ICPP , 2018
2018
Later among the works it cites.
N. Dryden et al. , “Aluminum: An asynchronous, GPU-aware communication library optimized for large-scale training of deep neural networks on HPC systems,” in MLHPC , 2018
2018
Later among the works it cites.
LLNL, “Lassen,” https://hpc.llnl.gov/hardware/platforms/lassen , 2018
2018
Later among the works it cites.
2018
Later among the works it cites.
——, “Exascale deep learning for climate analytics,” in SC , 2018
2018
Later among the works it cites.
A. Mathuriya et al. , “CosmoFlow: Using deep learning to learn the universe at scale,” in SC , 2018
2018
Later among the works it cites.
A. Gholami et al. , “Integrated model, batch and domain parallelism in training neural networks,” in SPAA , 2018
2018
Later among the works it cites.
Y. Oyama et al. , “Accelerating deep learning frameworks with micro-batches,” in CLUSTER , 2018
2018
Later among the works it cites.
2018
Later among the works it cites.
L. Wang et al. , “Superneurons: dynamic GPU memory management for training deep neural networks,” in PPoPP , 2018
2018
Later among the works it cites.