Fetching the paper…

Reading the bibliography…

2017

ChainerMN: Scalable Distributed Deep Learning Framework

Akiba, Takuya, Fukuda, Keisuke, Suzuki, Shuji

Original
PDF
Post on X

Understand

One of the keys for deep learning to have made a breakthrough in various fields was to utilize high computing powers centering around GPUs.

Enabling the use of further computing abilities by distributed processing is essential not only to make the deep learning bigger and faster but also to tackle unsolved challenges.
We present the design, implementation, and evaluation of ChainerMN, the distributed deep learning framework we have developed.
We demonstrate that ChainerMN can scale the learning process of the ResNet-50 model to the ImageNet dataset up to 128 GPUs with the parallel efficiency of 90%.

ChainerMN: Scalable Distributed Deep Learning Framework · Around

Built on

ImageNet: A Large-Scale Hierarchical Image Database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Chainer: a next-generation open source framework for deep learning
S. Tokui, K. Oono, S. Hido, and J. Clayton · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.

Similar

https://github.com/chainer/chainermn , 2017
ChainerMN · 2017
Cited alongside, same era.
https://developer.nvidia.com/nccl , 2017
NVIDIA Collective Communications Library (NCCL) · 2017
Cited alongside, same era.
http://www.gsic.titech.ac.jp/TSUBAME_ESJ , 11 2017
TSUBAME e-Science Journal · 2017
Cited alongside, same era.
The MNIST database of handwritten digits
Y. Lecun and C. Cortes
Cited in the paper.

Then

Accurate, large minibatch SGD: training imagenet in 1 hour
Original
P. Goyal, P. Dollár, R. B. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He · 2017
Closest in time.
Revisiting distributed synchronous sgd
X. Pan, J. Chen, R. Monga, S. Bengio, and R. Jozefowicz · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…