Layer-Parallel Training with GPU Concurrency of Deep Residual Neural Networks via Nonlinear Multigrid

D. J. Mavriplis, and A. Jameson, “Multigrid solution of the Navier-Stokes equations on triangular meshes,” AIAA journal 28, no. 8 (1990): 1415-1425

1990

Earlier work this paper cites.

D. J. Mavriplis, “Multigrid Techniques for Unstructured Mesh,” Numerical Methods for Fluid Dynamics V 5 (1995): 129

1995

Earlier work this paper cites.

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE 86, no. 11 (1998): 2278-2324

1998

Earlier work this paper cites.

W. Gropp, E. Lusk, A. Skjellum, Using MPI: portable parallel programming with the message-passing interface

1999

Earlier work this paper cites.

J. Nickolls, I. Buck, M. Garland, and K. Skadron, “Scalable parallel programming with CUDA,” Queue 6, no. 2 (2008): 40-53

2008

Earlier work this paper cites.

S. Chetlur, C. Woolley, P. Vandermersch, J. Cohen, J. Tran, B. Catanzaro, and E. Shelhamer, “cudnn: Efficient primitives for deep learning,” arXiv preprint arXiv:1410.0759 (2014)

Original

2014

Earlier work this paper cites.

R. D. Falgout, S. Friedhoff, T. V. Kolev, S. P. MacLachlan, and J. B. Schroder. “Parallel time integration with multigrid,” SIAM J. Sci. Comput., 36(6):C635–C661, 2014. LLNL-JRNL645325

2014

Earlier work this paper cites.

R. D. Falgout, S. Friedhoff, T. V. Kolev, S. P. MacLachlan, and J. B. Schroder. ”Parallel time integration with multigrid,” SIAM J. Sci. Comput., 36(6):C635–C661, 2014. LLNL-JRNL645325

2014

Earlier work this paper cites.

T. Bosse, , N. R. Gauger, A. Griewank, S. Günther, and V. Schulz, “One-shot approaches to design optimzation,” In Trends in PDE Constrained Optimization, pp. 43-66. Birkhäuser, Cham, 2014

2014

Earlier work this paper cites.

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and Li Fei-Fei, “Imagenet large scale visual recognition challenge,” International journal of computer vision 115, no. 3 (2015): 211-252

2015

Cited alongside, same era.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778. 2016

2016

Cited alongside, same era.

Y. Lu, A. Zhong, Q. Li, and B. Dong, “Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations,” arXiv preprint arXiv: 1710.10121 (2017)

2017

Cited alongside, same era.

V. A. Dobrev, T. Kolev, N. A. Petersson, and J. B. Schroder, “Two-level convergence theory for multigrid reduction in time (MGRIT),” SIAM Journal on Scientific Computing 39, no. 5 (2017): S501-S527

2017

Cited alongside, same era.

T. Ben-Nun and T. Hoefler, “Demystifying parallel and distributed deep learning: An in-depth concurrency analysis,” ACM Computing Surveys (CSUR) 52, no. 4 (2019): 1-43

2019

Later among the works it cites.

A. Reuther, P. Michaleas, M. Jones, V. Gadepally, S. Samsi, and J. Kepner, “Survey and benchmarking of machine learning accelerators,” arXiv preprint arXiv:1908.11348 (2019)

Original

2019

Later among the works it cites.

V. Gadepally, J. Goodwin, J. Kepner, A. Reuther, H. Reynolds, S. Samsi, J. Su, and D. Martinez, “AI Enabling Technologies: A Survey,” arXiv preprint arXiv: 1905.03592 (2019)

2019

Later among the works it cites.

S. Samsi, C. J. Mattioli, and M. S. Veillette, “Distributed Deep Learning for Precipitation Nowcasting,” arXiv preprint arXiv:1908.10964 (2019)

Original

2019

Later among the works it cites.