2018

A multiscale neural network based on hierarchical matrices

Fan, Yuwei, Lin, Lin, Ying, Lexing et al.

Understand

In this work we introduce a new multiscale artificial neural network based on the structure of $\mathcal{H}$-matrices.

This network generalizes the latter to the nonlinear case by introducing a local deep neural network at each spatial scale.
Numerical results indicate that the network is able to efficiently approximate discrete nonlinear maps obtained from discretized nonlinear partial differential equations, such as those arising from nonlinear Schr\"odinger equations and the Kohn-Sham density functional theory.

Built on

Vortex lines in an imperfect Bose gas
L. Pitaevskii · 1961
Earlier work this paper cites.
Inhomogeneous electron gas
P. Hohenberg and W. Kohn · 1964
Earlier work this paper cites.
Self-consistent equations including exchange and correlation effects
W. Kohn and L. J. Sham · 1965
Earlier work this paper cites.
Multi-level adaptive solutions to boundary-value problems
A. Brandt · 1977
Earlier work this paper cites.
A fast algorithm for particle simulations
L. Greengard and V. Rokhlin · 1987
Earlier work this paper cites.
Approximation capabilities of multilayer feedforward networks
K. Hornik · 1991
Earlier work this paper cites.
A sparse matrix arithmetic based on ℋ \mathcal{H} -matrices. part I: Introduction to ℋ \mathcal{H} -matrices
W. Hackbusch · 1999
Earlier work this paper cites.
A sparse ℋ \mathcal{H} -matrix arithmetic: general complexity estimates
W. Hackbusch and B. N. Khoromskij · 2000
Earlier work this paper cites.
Spectral Methods in MATLAB
L. Trefethen · 2000
Earlier work this paper cites.
Bose–Einstein condensation of atomic gases
J. R. Anglin and W. Ketterle · 2002
Earlier work this paper cites.
FMM and ℋ \mathcal{H} -matrices: a short introduction to the basic idea
M. Fenn and G. Steidl · 2002
Earlier work this paper cites.
An introduction to hierarchical matrices
W. Hackbusch, L. Grasedyck, and S. Börm · 2002
Earlier work this paper cites.
Introduction to hierarchical matrices with applications
S. Börm, L. Grasedyck, and W. Hackbusch · 2003
Earlier work this paper cites.
Computing the ground state solution of Bose–Einstein condensates by a normalized gradient flow
W. Bao and Q. Du · 2004
Earlier work this paper cites.
An “empirical interpolation” method: application to efficient reduced-basis discretization of partial differential equations
M. Barrault, Y. Maday, N. C. Nguyen, and A. T. Patera · 2004
Earlier work this paper cites.
Tomographic image reconstruction using artificial neural networks
P. Paschalis, N. D. Giokaris, A. Karabarbounis, G. Loudos, D. Maintas, C. Papanicolas, V. Spanoudaki, C. Tsoumpas, and E. Stiliaris · 2004
Earlier work this paper cites.
Multiscale finite element methods for porous media flows and their applications
Y. Efendiev and T. Hou · 2007
Earlier work this paper cites.
Efficient reduced-basis treatment of nonaffine and nonlinear partial differential equations
M. A. Grepl, Y. Maday, N. C. Nguyen, and A. T. Patera · 2007
Earlier work this paper cites.
A wavelet tour of signal processing: the sparse way
S. Mallat · 2008
Earlier work this paper cites.
Fast construction of hierarchical matrix representation from matrix–vector multiplication
L. Lin, J. Lu, and L. Ying · 2011
Earlier work this paper cites.
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury · 2012
Earlier work this paper cites.
ImageNet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.

Similar

Invariant scattering convolution networks
J. Bruna and S. Mallat · 2013
Cited alongside, same era.
Generalized multiscale finite element methods. nonlinear elliptic equations
Y. Efendiev, J. Galvis, G. Li, and M. Presho · 2014
Cited alongside, same era.
Deep learning of the tissue-regulated splicing code
M. K. K. Leung, H. Y. Xiong, L. J. Lee, and B. J. Frey · 2014
Cited alongside, same era.
A constrained backpropagation approach for the adaptive solution of partial differential equations
K. Rudd, G. D. Muro, and S. Ferrari · 2014
Cited alongside, same era.
Sequence to sequence learning with neural networks
I. Sutskever, O. Vinyals, and Q. V. Le · 2014
Cited alongside, same era.
Convolutional neural networks at constrained time cost
K. He and J. Sun · 2015
Cited alongside, same era.
Deep learning
Y. LeCun, Y. Bengio, and G. Hinton · 2015
Cited alongside, same era.
Deep neural nets as a method for quantitative structure-activity relationships
J. Ma, R. P. Sheridan, A. Liaw, G. E. Dahl, and V. Svetnik · 2015
Cited alongside, same era.
U-Net: Convolutional networks for biomedical image segmentation
O. Ronneberger, P. Fischer, and T. Brox · 2015
Cited alongside, same era.
Deep learning in neural networks: An overview
J. Schmidhuber · 2015
Cited alongside, same era.
Incorporating Nesterov momentum into Adam
D. Timothy · 2015
Cited alongside, same era.
Tensorflow: A system for large-scale machine learning
M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al · 2016
Cited alongside, same era.

Then

Partial differential equations for training deep neural networks
P. Chaudhari, A. Oberman, S. Osher, S. Soatto, and G. Carlier · 2017
Later among the works it cites.
Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
W. E, J. Han, and A. Jentzen · 2017
Later among the works it cites.
Solving parametric PDE problems with artificial neural networks
Original
Y. Khoo, J. Lu, and L. Ying · 2017
Later among the works it cites.
Expressive power of recurrent neural networks
Original
V. Khrulkov, A. Novikov, and I. Oseledets · 2017
Later among the works it cites.
Literature survey on low rank approximation of matrices
N. Kishore Kumar and J. Schneider · 2017
Later among the works it cites.
A survey on deep learning in medical image analysis
G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sánchez · 2017
Later among the works it cites.
Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising
K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang · 2017
Later among the works it cites.
Deep-learning tomography
M. Araya-Polo, J. Jennings, A. Adler, and T. Dahlke · 2018
Closest in time.
A unified deep artificial neural network approach to partial differential equations in complex geometries
J. Berg and K. Nyström · 2018
Closest in time.
DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs
L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2018
Closest in time.
Solving high-dimensional partial differential equations using deep learning
J. Han, A. Jentzen, and W. E · 2018
Closest in time.
Butterfly-Net: Optimal function representation based on convolutional neural networks
Original
Y. Li, X. Cheng, and J. Lu · 2018
Closest in time.
Forward-backward stochastic neural networks: Deep learning of high-dimensional partial differential equations
Original
R. M · 2018
Closest in time.
A mixed-scale dense convolutional neural network for image analysis
D. M. Pelt and J. A. Sethian · 2018
Closest in time.
Hidden physics models: Machine learning of nonlinear partial differential equations
M. Raissi and G. E. Karniadakis · 2018
Closest in time.
DGM: A deep learning algorithm for solving partial differential equations
J. Sirignano and K. Spiliopoulos · 2018
Closest in time.
Deep image prior
D. Ulyanov, A. Vedaldi, and V. Lempitsky · 2018
Closest in time.
Deep multiscale model learning
Original
Y. Wang, C. W. Siu, E. T. Chung, Y. Efendiev, and M. Wang · 2018
Closest in time.
DeePCG: constructing coarse-grained models via deep neural networks
Original
L. Zhang, J. Han, H. Wang, R. Car, and W. E · 2018
Closest in time.
Adaptive coupling of a deep neural network potential to a classical force field
Original
L. Zhang, H. Wang, and W. E · 2018
Closest in time.
A multiscale neural network based on hierarchical nested bases
Y. Fan, J. Feliu-Fabà, L. Lin, L. Ying, and L. Zepeda-Núñez · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…