2022

Message Passing Neural PDE Solvers

Brandstetter, Johannes, Worrall, Daniel, Welling, Max

Understand

The numerical solution of partial differential equations (PDEs) is difficult, having led to a century of research so far.

Recently, there have been pushes to build neural--numerical hybrid solvers, which piggy-backs the modern trend towards fully end-to-end learned systems.
Most works so far can only generalize over a subset of properties to which a generic solver would be faced, including: resolution, topology, geometry, boundary conditions, domain discretization regularity, dimensionality, etc.
In this work, we build a solver, satisfying these properties, where all the components are based on neural message passing, replacing all heuristically designed components in the computation graph with backprop-optimized neural function approximators.

Built on

Beitrag zur näherungsweisen integration totaler differentialgleichungen
Wilhelm Kutta · 1901
Earlier work this paper cites.
Coefficients for the study of runge-kutta integration processes
John Charles Butcher · 1963
Earlier work this paper cites.
On the partial difference equations of mathematical physics
Richard Courant, Kurt Otto Friedrichs, and Hans Lewy · 1967
Earlier work this paper cites.
An efficient integrator that uses gauss-radau spacings
Edgar Everhart · 1985
Earlier work this paper cites.
Spectral and finite difference solutions of the burgers equation
Claude Basdevant, Michel Deville, Pierre Haldenwang, Jean Lacroix, Jalil Ouazzani, R. Peyret, Paolo Orlandi, and Patera A.T · 1986
Earlier work this paper cites.
Symmetry groups of differential equations
P.J. Olver · 1986
Earlier work this paper cites.
The exponential accuracy of fourier and chebyshev differencing methods
Eitan Tadmor · 1986
Earlier work this paper cites.
The numerical analysis of ordinary differential equations: Runge-Kutta and general linear methods
John Charles Butcher · 1987
Earlier work this paper cites.
Solving Ordinary Differential Equations I (2nd Revised. Ed.): Nonstiff Problems
Ernst Hairer, Syvert Nørsett, and Gerhard Wanner · 1993
Earlier work this paper cites.
Efficient implementation of weighted eno schemes
Guang-Shan Jiang and Chi-Wang Shu · 1996
Earlier work this paper cites.
Neural operator: Graph kernel network for partial differential equations
Original
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar · 2003
Earlier work this paper cites.
High-order finite difference and finite volume weno schemes and discontinuous galerkin methods for cfd
C.-W. Shu · 2003
Earlier work this paper cites.
Multipole graph neural operator for parametric partial differential equations
Original
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar · 2006
Earlier work this paper cites.
The origins of computer weather prediction and climate modeling
Peter Lynch · 2008
Earlier work this paper cites.
Parametric design of aircraft geometry using partial differential equations
Michael Athanasopoulos, Hassan Ugail, and Gabriela Gonzalez · 2009
Earlier work this paper cites.
The graph neural network model
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini · 2009
Earlier work this paper cites.
Fourier neural operator for parametric partial differential equations
Original
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar · 2010
Earlier work this paper cites.

Similar

The numerical method of lines: integration of partial differential equations
William E Schiesser · 2012
Cited alongside, same era.
Introduction to partial differential equations
Peter J Olver · 2014
Cited alongside, same era.
Probabilistic ode solvers with runge-kutta means, 2014
Michael Schober, David Duvenaud, and Philipp Hennig · 2014
Cited alongside, same era.
Stability, consistency, and convergence of numerical discretizations
Douglas N Arnold · 2015
Cited alongside, same era.
Probabilistic numerics and uncertainty in computations
Philipp Hennig, Michael A. Osborne, and Mark Girolami · 2015
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Cited alongside, same era.
Numerical Approximation of Partial Differential Equations
Sören Bartels · 2016
Cited alongside, same era.
Fast and accurate deep network learning by exponential linear units (elus)
Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter · 2016
Cited alongside, same era.
Convolutional neural networks on graphs with fast localized spectral filtering
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst · 2016
Cited alongside, same era.
Convolutional neural networks for steady flow approximation
Xiaoxiao Guo, Wei Li, and Francesco Iorio · 2016
Cited alongside, same era.
Partial differential equations and stochastic methods in molecular dynamics
Tony Lelièvre and Gabriel Stoltz · 2016
Cited alongside, same era.
Instance normalization: The missing ingredient for fast stylization
Original
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky · 2016
Cited alongside, same era.

Then

Deep hidden physics models: Deep learning of nonlinear partial differential equations
Maziar Raissi · 2018
Later among the works it cites.
Dgm: A deep learning algorithm for solving partial differential equations
Justin Sirignano and Konstantinos Spiliopoulos · 2018
Later among the works it cites.
Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification
Yinhao Zhu and Nicholas Zabaras · 2018
Later among the works it cites.
Learning data-driven discretizations for partial differential equations
Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, and Michael P. Brenner · 2019
Later among the works it cites.
Prediction of aerodynamic flow fields using convolutional neural networks
Saakaar Bhatnagar, Yaser Afshar, Shaowu Pan, Karthik Duraisamy, and Shailendra Kaushik · 2019
Later among the works it cites.
Combining generative and discriminative models for hybrid inference
Victor Garcia Satorras, Zeynep Akata, and Max Welling · 2019
Later among the works it cites.
Learning to optimize multigrid PDE solvers
Daniel Greenfeld, Meirav Galun, Ronen Basri, Irad Yavneh, and Ron Kimmel · 2019
Later among the works it cites.
Learning neural PDE solvers with convergence guarantees
Original
Jun-Ting Hsieh, Shengjia Zhao, Stephan Eismann, Lucia Mirabella, and Stefano Ermon · 2019
Later among the works it cites.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al · 2019
Later among the works it cites.
Combining differentiable PDE solvers and graph neural networks for fluid flow prediction
Filipe De Avila Belbute-Peres, Thomas Economon, and Zico Kolter · 2020
Later among the works it cites.
Solving parametric pde problems with artificial neural networks
Yuehaw Khoo, Jianfeng Lu, and Lexing Ying · 2020
Later among the works it cites.
Learning to simulate complex physics with graph networks
Original
Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter W. Battaglia · 2020
Later among the works it cites.
Model reduction and neural networks for parametric pdes, 2021
Kaushik Bhattacharya, Bamdad Hosseini, Nikola B. Kovachki, and Andrew M. Stuart · 2021
Later among the works it cites.
Geometric and physical quantities improve e(3) equivariant message passing
Original
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik Bekkers, and Max Welling · 2021
Later among the works it cites.
Boundary graph neural networks for 3d simulations
Original
Andreas Mayr, Sebastian Lehner, Arno Mayrhofer, Christoph Kloss, Sepp Hochreiter, and Johannes Brandstetter · 2021
Later among the works it cites.
A physics-informed operator regression framework for extracting data-driven continuum models
Ravi G. Patel, Nathaniel A. Trask, Mitchell A. Wood, and Eric C. Cyr · 2021
Later among the works it cites.
E (n) equivariant graph neural networks
Original
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling · 2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…