Built on

E. J. Heller, Time-dependent approach to semiclassical dynamics, The Journal of Chemical Physics 62 (4) (1975) 1544–1555
1975
Earlier work this paper cites.
P. F. Brown, J. Cocke, S. A. Della Pietra, V. J. Della Pietra, F. Jelinek, J. Lafferty, R. L. Mercer, P. S. Roossin, A statistical approach to machine translation, Computational linguistics 16 (2) (1990) 79–85
1990
Earlier work this paper cites.
I. E. Lagaris, A. Likas, D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE transactions on neural networks 9 (5) (1998) 987–1000
1998
Earlier work this paper cites.
A. Rahimi, B. Recht, Random features for large-scale kernel machines, Advances in neural information processing systems 20 (2007)
2007
Earlier work this paper cites.
X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the thirteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceedings, 2010, pp. 249–256
2010
Earlier work this paper cites.
D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014)
Original
2014
Earlier work this paper cites.
K. B. Howell, Principles of Fourier analysis, CRC Press, 2016
2016
Earlier work this paper cites.
A. Vaswani, Attention is all you need, arXiv preprint arXiv:1706.03762 (2017)
Original
2017
Earlier work this paper cites.
J. Han, A. Jentzen, et al., Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Communications in mathematics and statistics 5 (4) (2017) 349–380
2017
Earlier work this paper cites.
J. Han, L. Zhang, R. Car, et al., Deep potential: A general representation of a many-body potential energy surface, arXiv preprint arXiv:1707.01478 (2017)
Original
2017
Earlier work this paper cites.
J. Devlin, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018)
Original
2018
Earlier work this paper cites.
B. Yu, et al., The deep ritz method: a deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics 6 (1) (2018) 1–12
2018
Earlier work this paper cites.
J. Han, A. Jentzen, W. E, Solving high-dimensional partial differential equations using deep learning, Proceedings of the National Academy of Sciences 115 (34) (2018) 8505–8510
2018
Earlier work this paper cites.
J. He, L. Li, J. Xu, C. Zheng, Relu deep neural networks and linear finite elements, arXiv preprint arXiv:1807.03973 (2018)
Original
2018
Earlier work this paper cites.
Z. J. Xu, Understanding training and generalization in deep learning by fourier analysis, arXiv preprint arXiv:1808.04295 (2018)
Original
2018
Earlier work this paper cites.
M. Deng, S. Li, G. Barbastathis, Learning to synthesize: splitting and recombining low and high spatial frequencies for image recovery, arXiv preprint arXiv:1811.07945 (2018)
Original
2018
Earlier work this paper cites.

Similar

J. Pan, S. Liu, D. Sun, J. Zhang, Y. Liu, J. Ren, Z. Li, J. Tang, H. Lu, Y.-W. Tai, et al., Learning dual convolutional neural networks for low-level vision, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3070–3079
2018
Cited alongside, same era.
A. Jacot, F. Gabriel, C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, Advances in neural information processing systems 31 (2018)
2018
Cited alongside, same era.
T. Tran, A. Hamilton, M. B. McKay, B. Quiring, P. S. Vassilevski, Dnn approximation of nonlinear finite element equations, arXiv preprint arXiv:1911.05240 (2019)
Original
2019
Cited alongside, same era.
Y. Liao, P. Ming, Deep nitsche method: Deep ritz method with essential boundary conditions, arXiv preprint arXiv:1912.01309 (2019)
Original
2019
Cited alongside, same era.

Then

C.-Y. Wu, R. Girshick, K. He, C. Feichtenhofer, P. Krahenbuhl, A multigrid method for efficiently training video models, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 153–162
2020
Later among the works it cites.
W. Cai, X. Li, L. Liu, A phase shift deep neural network for high frequency approximation and wave problems, SIAM Journal on Scientific Computing 42 (5) (2020) A3285–A3312
2020
Later among the works it cites.
Z. Liu, W. Cai, Z.-Q. J. Xu, Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains, arXiv preprint arXiv:2007.11207 (2020)
Original
2020
Later among the works it cites.
M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, R. Ng, Fourier features let networks learn high frequency functions in low dimensional domains, Advances in Neural Information Processing Systems 33 (2020) 7537–7547
2020

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Frequency-adaptive Multi-scale Deep Neural Networks

Huang, Jizu, You, Rukang, Zhou, Tao

Understand

Multi-scale deep neural networks (MscaleDNNs) with downing-scaling mapping have demonstrated superiority over traditional DNNs in approximating target functions characterized by high frequency features.

However, the performance of MscaleDNNs heavily depends on the parameters in the downing-scaling mapping, which limits their broader application.
In this work, we establish a fitting error bound to explain why MscaleDNNs are advantageous for approximating high frequency functions.
Building on this insight, we construct a hybrid feature embedding to enhance the accuracy and robustness of the downing-scaling mapping.

Built on

E. J. Heller, Time-dependent approach to semiclassical dynamics, The Journal of Chemical Physics 62 (4) (1975) 1544–1555
1975
Earlier work this paper cites.
P. F. Brown, J. Cocke, S. A. Della Pietra, V. J. Della Pietra, F. Jelinek, J. Lafferty, R. L. Mercer, P. S. Roossin, A statistical approach to machine translation, Computational linguistics 16 (2) (1990) 79–85
1990
Earlier work this paper cites.
I. E. Lagaris, A. Likas, D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE transactions on neural networks 9 (5) (1998) 987–1000
1998
Earlier work this paper cites.
A. Rahimi, B. Recht, Random features for large-scale kernel machines, Advances in neural information processing systems 20 (2007)
2007
Earlier work this paper cites.
X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the thirteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceedings, 2010, pp. 249–256
2010
Earlier work this paper cites.
D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014)
Original
2014
Earlier work this paper cites.
K. B. Howell, Principles of Fourier analysis, CRC Press, 2016
2016
Earlier work this paper cites.
A. Vaswani, Attention is all you need, arXiv preprint arXiv:1706.03762 (2017)
Original
2017
Earlier work this paper cites.
J. Han, A. Jentzen, et al., Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Communications in mathematics and statistics 5 (4) (2017) 349–380
2017
Earlier work this paper cites.
J. Han, L. Zhang, R. Car, et al., Deep potential: A general representation of a many-body potential energy surface, arXiv preprint arXiv:1707.01478 (2017)
Original
2017
Earlier work this paper cites.
J. Devlin, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018)
Original
2018
Earlier work this paper cites.
B. Yu, et al., The deep ritz method: a deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics 6 (1) (2018) 1–12
2018
Earlier work this paper cites.
J. Han, A. Jentzen, W. E, Solving high-dimensional partial differential equations using deep learning, Proceedings of the National Academy of Sciences 115 (34) (2018) 8505–8510
2018
Earlier work this paper cites.
J. He, L. Li, J. Xu, C. Zheng, Relu deep neural networks and linear finite elements, arXiv preprint arXiv:1807.03973 (2018)
Original
2018
Earlier work this paper cites.
Z. J. Xu, Understanding training and generalization in deep learning by fourier analysis, arXiv preprint arXiv:1808.04295 (2018)
Original
2018
Earlier work this paper cites.
M. Deng, S. Li, G. Barbastathis, Learning to synthesize: splitting and recombining low and high spatial frequencies for image recovery, arXiv preprint arXiv:1811.07945 (2018)
Original
2018
Earlier work this paper cites.

Similar

J. Pan, S. Liu, D. Sun, J. Zhang, Y. Liu, J. Ren, Z. Li, J. Tang, H. Lu, Y.-W. Tai, et al., Learning dual convolutional neural networks for low-level vision, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3070–3079
2018
Cited alongside, same era.
A. Jacot, F. Gabriel, C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, Advances in neural information processing systems 31 (2018)
2018
Cited alongside, same era.
T. Tran, A. Hamilton, M. B. McKay, B. Quiring, P. S. Vassilevski, Dnn approximation of nonlinear finite element equations, arXiv preprint arXiv:1911.05240 (2019)
Original
2019
Cited alongside, same era.
Y. Liao, P. Ming, Deep nitsche method: Deep ritz method with essential boundary conditions, arXiv preprint arXiv:1912.01309 (2019)
Original
2019
Cited alongside, same era.

Then

C.-Y. Wu, R. Girshick, K. He, C. Feichtenhofer, P. Krahenbuhl, A multigrid method for efficiently training video models, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 153–162
2020
Later among the works it cites.
W. Cai, X. Li, L. Liu, A phase shift deep neural network for high frequency approximation and wave problems, SIAM Journal on Scientific Computing 42 (5) (2020) A3285–A3312
2020
Later among the works it cites.
Z. Liu, W. Cai, Z.-Q. J. Xu, Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains, arXiv preprint arXiv:2007.11207 (2020)
Original
2020
Later among the works it cites.
M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, R. Ng, Fourier features let networks learn high frequency functions in low dimensional domains, Advances in Neural Information Processing Systems 33 (2020) 7537–7547
2020

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational physics 378 (2019) 686–707

2019

Cited alongside, same era.

C. M. Strofer, J.-L. Wu, H. Xiao, E. Paterson, Data-driven, physics-based feature extraction from fluid flow fields using convolutional neural networks, Communications in Computational Physics 25 (3) (2019) 625–650

2019

Cited alongside, same era.

N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, A. Courville, On the spectral bias of neural networks, in: International Conference on Machine Learning, PMLR, 2019, pp. 5301–5310

2019

Cited alongside, same era.

Z.-Q. J. Xu, Y. Zhang, T. Luo, Y. Xiao, Z. Ma, Frequency principle: Fourier analysis sheds light on deep neural networks, arXiv preprint arXiv:1901.06523 (2019)

Original

2019

Cited alongside, same era.

Z.-Q. J. Xu, Y. Zhang, Y. Xiao, Training behavior of deep neural network in frequency domain, in: Neural Information Processing: 26th International Conference, ICONIP 2019, Sydney, NSW, Australia, December 12–15, 2019, Proceedings, Part I 26, Springer, 2019, pp. 264–274

2019

Cited alongside, same era.

Y. Zhang, Z.-Q. J. Xu, T. Luo, Z. Ma, Explicitizing an implicit bias of the frequency principle in two-layer neural networks, arXiv preprint arXiv:1905.10264 (2019)

Original

2019

Cited alongside, same era.

Z. Shen, H. Yang, S. Zhang, Deep network approximation characterized by number of neurons. arxiv e-prints, page, arXiv preprint arXiv:1906.05497 (2019)

Original

2019

Cited alongside, same era.

S. Biland, V. C. Azevedo, B. Kim, B. Solenthaler, Frequency-aware reconstruction of fluid simulations with generative networks, arXiv preprint arXiv:1912.08776 (2019)

Original

2019

Cited alongside, same era.