Gossip training for deep learning

Neocognitron: A self-organizing neural network for a mechanism of pattern recognition unaffected by shift in position

Fukushima, K · 1980

Earlier work this paper cites.

Gradient based learning applied to document recognition. proceedings

LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P · 1998

Earlier work this paper cites.

Gossip-based computation of aggregate information

Kempe, D., Dobra, A., and Gehrke, J · 2003

Earlier work this paper cites.

Randomized gossip algorithms

Boyd, S., Ghosh, A., Prabhakar, B., and Shah, D · 2006

Earlier work this paper cites.

Learning multiple layers of features from tiny images

Krizhevsky, A · 2009

Earlier work this paper cites.

Distributed optimization and statistical learning via the alternating direction method of multipliers

Boyd, S., Parikh, N., Chu, E., Peleato, B., and Eckstein, J · 2011

Earlier work this paper cites.

Large scale distributed deep networks

Dean, J., Corrado, G. S., Monga, R., Chen, K., Devin, M., Le, Q. V., Mao, M. Z., Ranzato, M., Senior, A., Tucker, P., Yang, K., and Ng, A. Y · 2012

Cited alongside, same era.

Imagenet classification with deep convolutional neural networks

Krizhevsky, A., Sutskever, I., and Hinton, G. E · 2012

Cited alongside, same era.

Regularization of neural networks using dropconnect

Wan, L., Zeiler, M. D., Zhang, S., LeCun, Y., and Fergus, R · 2013

Cited alongside, same era.

The loss surfaces of multilayer networks

Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B., and LeCun, Y · 2015

Cited alongside, same era.

Asynchronous gossip principal components analysis

Fellus, J., Picard, D., and Gosselin, P.-H · 2015

Cited alongside, same era.

Microsoft coco: Common objects in context

Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C. L., and Dollár, P · 2015

Later among the works it cites.

Imagenet large scale visual recognition challeng

Russakovsky*, O., Deng*, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L · 2015

Later among the works it cites.

Deep learning with elastic averaging sgd

Original

Zhang, S., Choromanska, A., and LeCun, Y · 2015

Later among the works it cites.

Gossip dual averaging for decentralized optimization of pairwise functions

Colin, I., Bellet, A., Salmon, J., and Clemencon, S · 2016

Closest in time.

Deep residual learning for image recognition

He, K., Zhang, X., Ren, S., and Sun, J · 2016

Closest in time.