Fetching the paper…

Reading the bibliography…

Multi-scale Deep Neural Networks for Solving High Dimensional PDEs · Around

2019

Multi-scale Deep Neural Networks for Solving High Dimensional PDEs

Cai, Wei, Xu, Zhi-Qin John

Original
PDF
Post on X

Understand

In this paper, we propose the idea of radial scaling in frequency domain and activation functions with compact support to produce a multi-scale DNN (MscaleDNN), which will have the multi-scale capability in approximating high frequency and high dimensional functions and speeding up the solution of high dimensional PDEs.

Numerical results on high dimensional function fitting and solutions of high dimensional PDEs, using loss functions with either Ritz energy or least squared PDE residuals, have validated the increased power of multi-scale resolution and high frequency capturing of the proposed MscaleDNN.

Built on

Xu, Z.-Q. J., Zhang, Y., Luo, T., Xiao, Y. & Ma, Z. (2019), ‘Frequency principle: Fourier analysis sheds light on deep neural networks’, arXiv preprint arXiv:1901.06523
Original
1901
Earlier work this paper cites.
Zhang, Y., Xu, Z.-Q. J., Luo, T. & Ma, Z. (2019), ‘Explicitizing an implicit bias of the frequency principle in two-layer neural networks’, arXiv preprint arXiv:1905.10264
Original
1905
Earlier work this paper cites.
Luo, T., Ma, Z., Xu, Z.-Q. J. & Zhang, Y. (2019), ‘Theory of the frequency principle for general deep neural networks’, arXiv preprint arXiv:1906.09235
Original
1906
Earlier work this paper cites.
Cai, W., Li, X. & Liu, L. (2019), ‘A phase shift deep neural network for high frequency wave equations in inhomogeneous media’, Arxiv preprint, arXiv:1909.11759
Original
1909
Earlier work this paper cites.
Huang, J., Wang, H. & Yang, H. (2019), ‘Int-deep: A deep learning initialized iterative method for nonlinear problems’, arXiv preprint arXiv:1910.01594
Original
1910
Earlier work this paper cites.
Daubechies, I. (1992), Ten lectures on wavelets
1992
Earlier work this paper cites.
Glorot, X. & Bengio, Y. (2010), Understanding the difficulty of training deep feedforward neural networks, in
2010
Earlier work this paper cites.

Similar

Kingma, D. P. & Ba, J. (2014), ‘Adam: A method for stochastic optimization’, arXiv preprint arXiv:1412.6980
Original
2014
Cited alongside, same era.
Han, J. & E, W. (2016), ‘Deep learning approximation for stochastic control problems’, arXiv:1611.07422
Original
2016
Cited alongside, same era.
E, W., Han, J. & Jentzen, A. (2017), ‘Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations’, Communications in Mathematics and Statistics
2017
Cited alongside, same era.
Han, J., Zhang, L., Car, R. et al. (2017), ‘Deep potential: A general representation of a many-body potential energy surface’, arXiv preprint arXiv:1707.01478
Original
2017
Cited alongside, same era.

Then

He, J., Li, L., Xu, J. & Zheng, C. (2018), ‘Relu deep neural networks and linear finite elements’, arXiv preprint arXiv:1807.03973
Original
2018
Later among the works it cites.
Jacot, A., Gabriel, F. & Hongler, C. (2018), Neural tangent kernel: Convergence and generalization in neural networks, in
2018
Later among the works it cites.
Rahaman, N., Arpit, D., Baratin, A., Draxler, F., Lin, M., Hamprecht, F. A., Bengio, Y. & Courville, A. (2018), ‘On the spectral bias of deep neural networks’, arXiv preprint arXiv:1806.08734
Original
2018
Later among the works it cites.
Rotskoff, G. & Vanden-Eijnden, E. (2018), Parameters as interacting particles: long time convergence and asymptotic error scaling of neural networks, in
2018
Later among the works it cites.
Xu, Z. J. (2018), ‘Understanding training and generalization in deep learning by fourier analysis’, arXiv preprint arXiv:1808.04295

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Khoo, Y., Lu, J. & Ying, L. (2017), ‘Solving parametric pde problems with artificial neural networks’, arXiv preprint arXiv:1707.03351

2017

Cited alongside, same era.

E, W. & Yu, B. (2018), ‘The deep ritz method: A deep learning-based numerical algorithm for solving variational problems’, Communications in Mathematics and Statistics

2018

Cited alongside, same era.

Fan, Y., Lin, L., Ying, L. & Zepeda-Núnez, L. (2018), ‘A multiscale neural network based on hierarchical matrices’, arXiv preprint arXiv:1807.01883

2018

Cited alongside, same era.

2018

Later among the works it cites.

Xu, Z.-Q. J., Zhang, Y. & Xiao, Y. (2018), ‘Training behavior of deep neural network in frequency domain’, arXiv preprint arXiv:1807.01251

2018

Later among the works it cites.