Fetching the paper…
Reading the bibliography…
In this review, we survey the latest approaches and techniques developed to overcome the spectral bias towards low frequency of deep neural network learning methods in learning multiple-frequency solutions of partial differential equations.
C. E. Shannon, A mathematical theory of communication, The Bell system technical journal 27 (3) (1948) 379–423
1948
Earlier work this paper cites.
T. E. Duncan, On the calculation of mutual information, SIAM Journal on Applied Mathematics 19 (1) (1970) 215–220
1970
Earlier work this paper cites.
H. Tennekes, J. Lumley, A first course in turbulence, MIT Press, 1972
1972
Earlier work this paper cites.
A. Brandt, Multi-level adaptive solutions to boundary-value problems, Mathematics of computation 31 (138) (1977) 333–390
1977
Earlier work this paper cites.
M. Y. Hussaini, T. A. Zang, Spectral methods in fluid dynamics, Annual review of fluid mechanics 19 (1) (1987) 339–367
1987
Earlier work this paper cites.
G. Cybenko, Approximation by superpositions of a sigmoidal function, Mathematics of control, signals and systems 2 (4) (1989) 303–314
1989
Earlier work this paper cites.
K. Hornik, M. Stinchcombe, H. White, Multilayer feedforward networks are universal approximators, Neural networks 2 (5) (1989) 359–366
1989
Earlier work this paper cites.
K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural networks 4 (2) (1991) 251–257
1991
Earlier work this paper cites.
J. Xu, Iterative methods by space decomposition and subspace correction, SIAM review 34 (4) (1992) 581–613
1992
Earlier work this paper cites.
M. Leshno, V. Y. Lin, A. Pinkus, S. Schocken, Multilayer feedforward networks with a nonpolynomial activation function can approximate any function, Neural networks 6 (6) (1993) 861–867
1993
Earlier work this paper cites.
M. Dissanayake, N. Phan-Thien, Neural-network-based approximations for solving partial differential equations, communications in Numerical Methods in Engineering 10 (3) (1994) 195–201
1994
Earlier work this paper cites.
W. Hackbusch, The integral equation method, Birkhäuser Basel, 1995
1995
Earlier work this paper cites.
O. Axelsson, Iterative solution methods, Cambridge university press, 1996
1996
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324
1998
Earlier work this paper cites.
B.-n. Jiang, The least-squares finite element method: theory and applications in computational fluid dynamics and electromagnetics, Springer Science & Business Media, 1998
1998
Earlier work this paper cites.
I. Lagaris, A. Likas, D. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE transactions on Neural Networks 9 (5) (1998) 987–1000
1998
Earlier work this paper cites.
I. Lagaris, A. Likas, G. Papageorgiou, Neural-network methods for boundary value problems with irregular boundaries, IEEE transactions on Neural Networks 11 (5) (2000) 1041–1049
2000
Earlier work this paper cites.
Z. Liu, W. Cai, Z.-Q. J. Xu, Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains, Communications in Computational Physics 28 (5) (2020) 1970–2001
2001
Earlier work this paper cites.
P. Ciarlet, The finite element method for elliptic problems, Society for Industrial and Applied Mathematics, 2002
2002
Earlier work this paper cites.
Y. Saad, Iterative methods for sparse linear systems, SIAM, 2003
2003
Earlier work this paper cites.
J. C. Strikwerda, Finite difference schemes and partial differential equations, SIAM, 2004
2004
Earlier work this paper cites.
O. Zienkiewicz, R. Taylor, The finite element method for solid and structural mechanics, Elsevier, 2005
2005
Earlier work this paper cites.
I. Livshits, A. Brandt, Accuracy properties of the wave-ray multigrid algorithm for helmholtz equations, SIAM J. Sci. Comput. 28 (2006) 1228–1251
2006
Earlier work this paper cites.
Y. Bengio, et al., Learning deep architectures for ai, Foundations and trends® in Machine Learning 2 (1) (2009) 1–127
2009
Earlier work this paper cites.
P. B. Bochev, M. D. Gunzburger, Least-squares finite element methods, Vol. 166, Springer Science & Business Media, 2009
2009
Earlier work this paper cites.
J. Shen, T. Tang, L.-L. Wang, Spectral methods: algorithms, analysis and applications, Vol. 41, Springer Science & Business Media, 2011
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, Vol. 25, 2012, pp. 1106–1114
2012
Earlier work this paper cites.
G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al., Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, IEEE Signal processing magazine 29 (6) (2012) 82–97
2012
Earlier work this paper cites.
W. Cai, Computational Methods for Electromagnetic Phenomena: electrostatics in solvation, scattering, and electron transport, Cambridge University Press, 2013
2013
Earlier work this paper cites.
A. J. Chorin, Vorticity and turbulence, Springer Science & Business Media, 2013
2013
Earlier work this paper cites.
M. Born, E. Wolf, Principles of optics: electromagnetic theory of propagation, interference and diffraction of light, Elsevier, 2013
2013
Earlier work this paper cites.
L. D. Landau, E. M. Lifshitz, Quantum mechanics: non-relativistic theory, Vol. 3, Elsevier, 2013
2013
Earlier work this paper cites.
S. Olver, A. Townsend, A fast and well-conditioned spectral method, SIAM review 55 (3) (2013) 462–489
2013
Earlier work this paper cites.
W. Hackbusch, Multi-grid methods and applications, Vol. 4, Springer Science and Business Media, 2013
2013
Earlier work this paper cites.
Y. LeCun, Y. Bengio, G. Hinton, Deep learning, nature 521 (7553) (2015) 436–444
2015
Earlier work this paper cites.
N. Tishby, N. Zaslavsky, Deep learning and the information bottleneck principle, in: 2015 ieee information theory workshop (itw), IEEE, 2015, pp. 1–5
2015
Earlier work this paper cites.
D. Hendrycks, K. Gimpel, Gaussian error linear units (gelus), arXiv preprint arXiv:1606.08415 (2016)
2016
Earlier work this paper cites.
W. E, J. Han, A. Jentzen, Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Communications in Mathematics and Statistics 5 (4) (2017) 349–380
2017
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Advances in neural information processing systems 30 (2017)
2017
Earlier work this paper cites.
J. Han, L. Zhang, R. Car, et al., Deep potential: A general representation of a many-body potential energy surface, Communications in Computational Physics 23 (3) (2018)
2018
Earlier work this paper cites.
J. Rammer, Quantum transport theory, CRC Press, 2018
2018
Earlier work this paper cites.
J. Sirignano, K. Spiliopoulos, Dgm: A deep learning algorithm for solving partial differential equations, Journal of computational physics 375 (2018) 1339–1364
2018
Earlier work this paper cites.
J. Berg, K. Nyström, A unified deep artificial neural network approach to partial differential equations in complex geometries, Neurocomputing 317 (2018) 28–41
2018
Earlier work this paper cites.
W. E, B. Yu, The deep ritz method: A deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics 6 (1) (2018) 1–12
2018
Earlier work this paper cites.
2018
Earlier work this paper cites.
A. Jacot, F. Gabriel, C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, Advances in neural information processing systems 31 (2018) 8580–8589
2018
Earlier work this paper cites.
M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378 (2019) 686–707
2019
Earlier work this paper cites.
B. E. Saleh, M. C. Teich, Fundamentals of photonics, john Wiley & sons, 2019
2019
Earlier work this paper cites.
N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, A. Courville, On the spectral bias of neural networks, in: International Conference on Machine Learning, PMLR, 2019, pp. 5301–5310
2019
Earlier work this paper cites.
Z.-Q. J. Xu, Y. Zhang, Y. Xiao, Training behavior of deep neural network in frequency domain, International Conference on Neural Information Processing (2019) 264–274
2019
Cited alongside, same era.
B. Ronen, D. Jacobs, Y. Kasten, S. Kritchman, The convergence rate of neural networks for learned functions of different frequencies, Advances in Neural Information Processing Systems 32 (2019) 4761–4771
2019
Cited alongside, same era.
2019
Cited alongside, same era.
J. Lee, L. Xiao, S. Schoenholz, Y. Bahri, R. Novak, J. Sohl-Dickstein, J. Pennington, Wide neural networks of any depth evolve as linear models under gradient descent, Advances in neural information processing systems 32 (2019)
2019
Cited alongside, same era.
T. Luo, Z. Ma, Z. Wang, Z. J. Xu, Y. Zhang, An upper limit of decaying rate with respect to frequency in linear frequency principle model, in: Mathematical and Scientific Machine Learning, PMLR, 2022, pp. 205–214
2022
Later among the works it cites.
2022
Later among the works it cites.
W. E, C. Ma, L. Wu, The barron space and the flow-induced function spaces for neural network models, Constructive Approximation 55 (1) (2022) 369–406
2022
Later among the works it cites.
T. Luo, Z. Ma, Z.-Q. J. Xu, Y. Zhang, On the exact computation of linear frequency principle dynamics and its generalization, SIAM Journal on Mathematics of Data Science 4 (4) (2022) 1272–1292
2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2019
Cited alongside, same era.
J. He, J. Xu, Mgnet: A unified framework of multigrid and convolutional neural network, Science china mathematics 62 (2019) 1331–1354
2019
Cited alongside, same era.
W. E, Machine learning and computational mathematics, Communications in Computational Physics 28 (5) (2020) 1639–1670
2020
Cited alongside, same era.
J. Hermann, Z. Schätzle, F. Noé, Deep-neural-network solution of the electronic schrödinger equation, Nature Chemistry 12 (10) (2020) 891–897
2020
Cited alongside, same era.
D. Pfau, J. S. Spencer, A. G. Matthews, W. M. C. Foulkes, Ab initio solution of the many-electron schrödinger equation with deep neural networks, Physical Review Research 2 (3) (2020) 033429
2020
Cited alongside, same era.
Z.-Q. J. Xu, Y. Zhang, T. Luo, Y. Xiao, Z. Ma, Frequency principle: Fourier analysis sheds light on deep neural networks, Communications in Computational Physics 28 (5) (2020) 1746–1767
2020
Cited alongside, same era.
M. L. Z. Cai, J. Chen, X. Liu, Deep least-squares methods: An unsupervised learning-based numerical method for solving elliptic pdes, Journal of Computational Physics. (2020)
2020
Cited alongside, same era.
B. Wang, W. Zhang, W. Cai, Multi-scale deep neural network (mscalednn) methods for oscillatory stokes flows in complex domains, Communications in Computational Physics 28 (5) (2020) 2139–2157
2020
Cited alongside, same era.
J. Chen, X. Chi, Z. Yang, et al., Bridging traditional and machine learning-based algorithms for solving pdes: The random feature method, Journal of Machine Learning (2022)
2022
Later among the works it cites.
X. Huang, H. Liu, B. Shi, Z. Wang, K. Yang, Y. Li, M. Wang, H. Chu, J. Zhou, F. Yu, B. Dong, L. Chen, A universal pinns method for solving partial differential equations with a point source, in: International Joint Conference on Artificial Intelligence, 2022
2022
Later among the works it cites.
2022
Later among the works it cites.
A. D. Jagtap, Y. Shin, K. Kawaguchi, G. E. Karniadakis, Deep kronecker neural networks: A general framework for neural networks with adaptive activation functions, Neurocomputing 468 (2022) 165–180
2022
Later among the works it cites.
Y. Chen, B. Dong, J. Xu, Meta-mgnet: Meta multigrid networks for solving parameterized partial differential equations, Journal of computational physics 455 (2022) 110996
2022
Later among the works it cites.
2022
Later among the works it cites.
2022
Later among the works it cites.
H. Zhou, Z. Qixuan, Z. Jin, T. Luo, Y. Zhang, Z.-Q. Xu, Empirical phase diagram for three-layer neural networks with infinite width, Advances in Neural Information Processing Systems 35 (2022) 26021–26033
2022
Later among the works it cites.
Z. Zhang, Z.-Q. J. Xu, Implicit regularization of dropout, arXiv preprint arXiv:2207.05952 (2022)
2022
Later among the works it cites.
H. Zhou, Q. Zhou, T. Luo, Y. Zhang, Z.-Q. Xu, Towards understanding the condensation of neural networks at initial training, Advances in Neural Information Processing Systems 35 (2022) 2184–2196
2022
Later among the works it cites.
S. Mishra, R. Molinaro, Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems, IMA J. Numer. Anal 42 (2) (2022) 981–1022
2022
Later among the works it cites.
J. Lu, Y. Lu, A priori generalization error analysis of two-layer neural networks for solving high dimensional schr¨odinger eigenvalue problems, Commun. AMS 2 (1) (2022) 1–21
2022
Later among the works it cites.
P. Dondl, J. Müller, M. Zeinhofer, Uniform convergence guarantees for the deep ritz method for nonlinear problems, Advances in Continuous and Discrete Models (2022) 1–9
2022
Later among the works it cites.
S. Lanthaler, S. Mishra, G. Karniadakis, Error estimates for deeponets: A deep learning framework in infinite dimensions, Transactions of Mathematics and Its Applications 6 (1:tnac001) (2022)
2022
Later among the works it cites.
arXiv:https://doi.org/10.1137/22M1527763
Z. Gao, L. Yan, T. Zhou, Failure-informed adaptive sampling for pinns , SIAM Journal on Scientific Computing 45 (4) (2023) A1971–A1994 · 2023
Later among the works it cites.
Y. Kim, S. Kim, I. Seo, B. Shin, Phase-shifted adversarial training, Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence 216 (2023) 1068–1077
2023
Later among the works it cites.
L. Zhang, W. Cai, Z.-Q. J. Xu, A correction and comments on “multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains cicp, 28 (5): 1970–2001, 2020”, Communications in Computational Physics 33 (5) (2023) 1509–1513
2023
Later among the works it cites.
M. Chen, R. Niu, W. Zheng, Adaptive multi-scale neural network with resnet blocks for solving partial differential equations, Nonlinear Dynamics 111 (7) (2023) 6499–6518
2023
Later among the works it cites.
Z. Wang, M. Chen, J. Chen, Solving multiscale elliptic problems by sparse radial basis function neural networks, Journal of Computational Physics 492 (2023) 112452
2023
Later among the works it cites.
X.-A. Li, Z.-Q. J. Xu, L. Zhang, Subspace decomposition based dnn algorithm for elliptic type multi-scale pdes, Journal of Computational Physics 488 (2023) 112242
2023
Later among the works it cites.
J. Ying, J. Liu, J. Chen, S. Cao, M. Hou, Y. Chen, Multi-scale fusion network: A new deep learning structure for elliptic interface problems, Applied Mathematical Modelling 114 (2023) 252–269
2023
Later among the works it cites.
S. Li, Y. Xia, Y. Liu, Q. Liao, A deep domain decomposition method based on fourier features, Journal of Computational and Applied Mathematics 423 (2023) 114963
2023
Later among the works it cites.
2023
Later among the works it cites.
A. D. Jagtap, G. E. Karniadakis, How important are activation functions in regression and classification? a survey, performance comparison, and future directions, Journal of Machine Learning for Modeling and Computing 4 (1) (2023)
2023
Later among the works it cites.
2023
Later among the works it cites.
Y. Xu, Multi-grade deep learning, arXiv preprint arXiv:2302.00150 (2023)
2023
Later among the works it cites.
2023
Later among the works it cites.
2023
Later among the works it cites.
2023
Later among the works it cites.
2023
Later among the works it cites.
2023
Later among the works it cites.
Y. Shin, Z. Zhang, G. Karniadakis, Error estimates of residual minimization using neural networks for linear pdes, Journal of Machine Learning for Modeling and Computing 4 (4) (2023)
2023
Later among the works it cites.
2023
Later among the works it cites.
L. Liu, K. Nath, W. Cai, A causality-deeponet for causal responses of linear dynamical systems, Commun. Comput. Phys. 35 (5) (2024) 1194–1228
2024
Later among the works it cites.
Z. Gao, T. Tang, L. Yan, et al., Failure-informed adaptive sampling for pinns, part ii: combining with re-sampling and subset simulation, Communications on Applied Mathematics and Computation 6 (3) (2024) 1720–1741
2024
Later among the works it cites.
2024
Later among the works it cites.
2024
Later among the works it cites.
E. Zhang, A. Kahana, A. Kopanicakova, E. Turkel, R. Ranade, J. Pathak, G. Karniadakis, Blending neural operators and relaxation methods in pde numerical solvers, Nat Mach Intell 6 (2024) 1303–1313
2024
Later among the works it cites.
Z. Aldirany, R. Cottereau, M. Laforest, S. Prudhomme, Multi-level neural networks for accurate solutions of boundary-value problems, Computer Methods in Applied Mechanics and Engineering 419 (2024) 116666
2024
Later among the works it cites.
2024
Later among the works it cites.
2024
Later among the works it cites.
H. Wu, K. Zhang, D. Zhou, W.-L. Chen, Z. Han, Y. Cao, High-flexibility reconstruction of small-scale motions in wall turbulence using a generalized zero-shot learning, Journal of Fluid Mechanics 990 (2024) R1
2024
Later among the works it cites.
Y. Chen, D. Wang, D. Feng, G. Tian, V. Gupta, R. Cao, M. Wan, S. Chen, Three-dimensional spatiotemporal wind field reconstruction based on lidar and multi-scale pinn, Applied Energy 377 (2025) 124577
2025
Closest in time.
2025
Closest in time.
Y. Shin, J. Darbon, G. Karniadakis, On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type pdes, COMMUNICATIONS IN COMPUTATIONAL PHYSICS 28 (5) (2020) 2042–74
2042
Closest in time.