Multilevel Initialization for Layer-Parallel Deep Neural Network Training

DOI doi:/10.4231/R7RX991C

Baumgardner, M.F., Biehl, L.L., Landgrebe, D.A.: 220 band aviris hyperspectral image data set: June 12, 1992 indian pine test site 3 (2015) · 1947

Earlier work this paper cites.

Lions, J.L.: Optimal control of systems governed by partial differential equations (1971)

1971

Earlier work this paper cites.

BIT Numerical Mathematics 12

Kronsjö, L., Dahlquist, G.: On the design of nested iterations for elliptic difference equations · 1972

Earlier work this paper cites.

BIT Numerical Mathematics 15

Kronsjö, L.: A note on the nested iterations method · 1975

Earlier work this paper cites.

Beiträge Numer. Math 9

Hackbusch, W.: On the convergence of multi-grid iterations · 1981

Earlier work this paper cites.

SIAM, Philadelphia, PA, USA (2000)

Briggs, W.L., Henson, V.E., McCormick, S.F.: A multigrid tutorial, 2nd edn · 2000

Earlier work this paper cites.

Academic Press, London, UK (2001)

Trottenberg, U., Oosterlee, C., Sch u ¨ \ddot{\mbox{u}} ller, A.: Multigrid · 2001

Earlier work this paper cites.

In: L.T. Biegler, M. Heinkenschloss, O. Ghattas, B. van Bloemen Waanders (eds.) Large-Scale PDE-Constrained Optimization, pp. 3–13. Springer Berlin Heidelberg (2003)

Biegler, L.T., Ghattas, O., Heinkenschloss, M., van Bloemen Waanders, B.: Large-scale pde-constrained optimization: An introduction · 2003

Earlier work this paper cites.

Numerical Linear Algebra with Applications 15

De Sterck, H., Manteuffel, T., McCormick, S., Nolting, J., Ruge, J., Tang, L.: Efficiency-based h h -and h ​ p hp -refinement strategies for finite element methods · 2008

Earlier work this paper cites.

Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics 9

Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks · 2010

Cited alongside, same era.

American Mathematical Soc. (2010)

Tröltzsch, F.: Optimal control of partial differential equations: theory, methods, and applications, vol. 112 · 2010

Cited alongside, same era.

SIAM Journal on Scientific Computing 33

Adler, J., Manteuffel, T.A., McCormick, S.F., Nolting, J., Ruge, J.W., Tang, L.: Efficiency based adaptive local refinement for first-order system least-squares formulations · 2011

Cited alongside, same era.

In: Proceedings of the IEEE international conference on computer vision, pp. 1026–1034 (2015)

He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification · 2015

Cited alongside, same era.

In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition · 2016

Cited alongside, same era.

In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

Chang, B., Meng, L., Haber, E., Ruthotto, L., Begert, D., Holtham, E.: Reversible architectures for arbitrarily deep residual neural networks · 2018

Later among the works it cites.

Research in the Mathematical Sciences 5

Chaudhari, P., Oberman, A., Osher, S., Soatto, S., Carlier, G.: Deep relaxation: partial differential equations for optimizing deep neural networks · 2018

Later among the works it cites.

In: Advances in neural information processing systems, pp. 6571–6583 (2018)

Chen, T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations · 2018

Later among the works it cites.

In: Advances in Neural Information Processing Systems, pp. 571–581 (2018)

Hanin, B., Rolnick, D.: How to start training: The effect of initialization and architecture · 2018

Later among the works it cites.

IEEE transactions on neural networks and learning systems 30

Humbird, K.D., Peterson, J.L., McClarren, R.G.: Deep neural network initialization with decision trees · 2018

Later among the works it cites.