Applying machine learning techniques to the quickly growing data in science and industry requires highly-scalable algorithms.
Reading the bibliography…
Two algorithms for barrier synchronization
Hensgen, D., Finkel, R., and Manber, U · 1988
Earlier work this paper cites.
The atis spoken language systems pilot corpus
Hemphill, C. T., Godfrey, J. J., Doddington, G. R., et al · 1990
Earlier work this paper cites.
Long short-term memory
Hochreiter, S., and Schmidhuber, J · 1997
Earlier work this paper cites.
Improving the performance of collective operations in mpich
Thakur, R., and Gropp, W. D · 2003
Earlier work this paper cites.
Optimization of collective reduction operations
Rabenseifner, R · 2004
Earlier work this paper cites.
Introducing the webb spam corpus: Using email spam to identify web spam automatically
Webb, S., Caverlee, J., and Pu, C · 2006
Earlier work this paper cites.
Collective communication: theory, practice, and experience
Chan, E., Heimlich, M., Purkayastha, A., and Van De Geijn, R · 2007
Earlier work this paper cites.
Implementation and performance analysis of non-blocking collective operations for mpi
Hoefler, T., Lumsdaine, A., and Rehm, W · 2007
Earlier work this paper cites.
Implementation and Performance Analysis of Non-Blocking Collective Operations for MPI
Hoefler, T., Lumsdaine, A., and Rehm, W · 2007
Earlier work this paper cites.
Message Progression in Parallel Computing - To Thread or not to Thread?
Hoefler, T., and Lumsdaine, A · 2008
Earlier work this paper cites.
Mpi reduction operations for sparse floating-point data
Hofmann, M., and Rünger, G · 2008
Earlier work this paper cites.
Identifying suspicious urls: an application of large-scale online learning
Ma, J., Saul, L. K., Savage, S., and Voelker, G. M · 2009
Earlier work this paper cites.
Transparent neutral element elimination in mpi reduction operations
Träff, J · 2010
Earlier work this paper cites.
MPI: A Message-Passing Interface Standard Version 3.0, 09 2012
Forum, M. P. I · 2012
Earlier work this paper cites.
Project adam: Building an efficient and scalable deep learning training system
Chilimbi, T. M., Suzue, Y., Apacible, J., and Kalyanaraman, K · 2014
Earlier work this paper cites.
Using Advanced MPI: Modern Features of the Message-Passing Interface
Gropp, W., Hoefler, T., Thakur, R., and Lusk, E · 2014
Earlier work this paper cites.
Energy, Memory, and Runtime Tradeoffs for Implementing Collective Communication Operations
Hoefler, T., and Moor, D · 2014
Earlier work this paper cites.
Communication-efficient distributed dual coordinate ascent
Jaggi, M., Smith, V., Takác, M., Terhorst, J., Krishnan, S., Hofmann, T., and Jordan, M. I · 2014
Earlier work this paper cites.
Scaling distributed machine learning with the parameter server
Li, M., Andersen, D. G., Park, J. W., Smola, A. J., Ahmed, A., Josifovski, V., Long, J., Shekita, E. J., and Su, B.-Y · 2014
Earlier work this paper cites.
1-bit Stochastic Gradient Descent and its Application to Data-parallel Distributed Training of Speech DNNs
Seide, F., Fu, H., Droppo, J., Li, G., and Yu, D · 2014
Earlier work this paper cites.
An introduction to computational networks and the computational network toolkit
Yu, D., Eversole, A., Seltzer, M., Yao, K., Huang, Z., Guenter, B., Kuchaiev, O., Zhang, Y., Seide, F., Wang, H., et al · 2014
Earlier work this paper cites.
Kylix: A sparse allreduce for commodity clusters
Zhao, H., and Canny, J · 2014
Cited alongside, same era.
Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., and Zhang, Z · 2015
Cited alongside, same era.
Taming the wild: A unified analysis of Hogwild
De Sa, C., Zhang, C., Olukotun, K., and Ré, C · 2015
Cited alongside, same era.
Asynchronous parallel stochastic gradient for nonconvex optimization
Lian, X., Huang, Y., Li, Y., and Liu, J · 2015
Cited alongside, same era.
Asynchronous stochastic coordinate descent: Parallelism and convergence properties
Liu, J., and Wright, S. J · 2015
Cited alongside, same era.
Imagenet large scale visual recognition challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al · 2015
Cited alongside, same era.
Scalable distributed dnn training using commodity gpu cloud computing
Strom, N · 2015
Cited alongside, same era.
Lbann: Livermore big artificial neural network hpc toolkit
Van Essen, B., Kim, H., Pearce, R., Boakye, K., and Chen, B · 2015
Cited alongside, same era.
Coordinate descent algorithms
Wright, S. J · 2015
Cited alongside, same era.
Petuum: A new platform for distributed machine learning on big data
Xing, E. P., Ho, Q., Dai, W., Kim, J. K., Wei, J., Lee, S., Zheng, X., Xie, P., Kumar, A., and Yu, Y · 2015
Cited alongside, same era.
Tensorflow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al · 2016
Cited alongside, same era.
Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering
Chen, K., and Huo, Q · 2016
Cited alongside, same era.
Codreanu, V., Podareanu, D., and Saletore, V · 2017
Later among the works it cites.
Accurate, large minibatch sgd: Training imagenet in 1 hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., and He, K · 2017
Later among the works it cites.
Introduction to pytorch
Ketkar, N · 2017
Later among the works it cites.
Deep gradient compression: Reducing the communication bandwidth for distributed training
Lin, Y., Han, S., Mao, H., Wang, Y., and Dally, W. J · 2017
Later among the works it cites.
Aligned Hansards of the 36th Parliament of Canada, October 2017
of the USC Information Sciences Institute, N. L. G · 2017
Later among the works it cites.
meprop: Sparsified back propagation for accelerated deep learning with reduced overfitting
Sun, X., Ren, X., Ma, S., and Wang, H · 2017
Later among the works it cites.
Inception-v4, inception-resnet and the impact of residual connections on learning
Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. A · 2017
Later among the works it cites.
Terngrad: Ternary gradients to reduce communication in distributed deep learning
Wen, W., Xu, C., Yan, F., Wu, C., Wang, Y., Chen, Y., and Li, H · 2017
Later among the works it cites.
Scaling sgd batch size to 32k for imagenet training
You, Y., Gitman, I., and Ginsburg, B · 2017
Later among the works it cites.
Yellowfin and the art of momentum tuning
Zhang, J., Mitliagkas, I., and Ré, C · 2017
Later among the works it cites.
The convergence of sparsified gradient methods
Alistarh, D., Hoefler, T., Johansson, M., Konstantinov, N., Khirirat, S., and Renggli, C · 2018
Closest in time.
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
Ben-Nun, T., and Hoefler, T · 2018
Closest in time.
Apache Spark on the CSCS Cluster
Centre, C. S. N. S · 2018
Closest in time.
The CSCS Piz Daint supercomputer
Centre, C. S. N. S · 2018
Closest in time.
The effect of network width on the performance of large-batch training
Chen, L., Wang, H., Zhao, J., Papailiopoulos, D., and Koutris, P · 2018
Closest in time.
BERT: pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M., Lee, K., and Toutanova, K · 2018
Closest in time.
Synchronous multi-gpu training for deep learning with low-precision communications: An empirical study
Grubic, D., Tam, L., Alistarh, D., and Zhang, C · 2018
Closest in time.
Bert explained: State of the art language model for nlp
Horev, R · 2019
Closest in time.
Cntk examples
Microsoft · 2019
Closest in time.
Yet another accelerated sgd: Resnet-50 training on imagenet in 74.7 seconds
Yamazaki, M., Kasagi, A., Tabuchi, A., Honda, T., Miwa, M., Fukumoto, N., Tabaru, T., Ike, A., and Nakashima, K · 2019
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…