Fetching the paper…
Reading the bibliography…
Transformer has shown state-of-the-art performance on various applications and has recently emerged as a promising tool for surrogate modeling of partial differential equations (PDEs).
Numerical experiments in homogeneous turbulence
Robert S. Rogallo · 1981
Earlier work this paper cites.
Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems
Tianping Chen and Hong Chen · 1995
Earlier work this paper cites.
Domain decomposition methods for partial differential equations
Alfio Quarteroni and Alberto Valli · 1999
Earlier work this paper cites.
Neural operator: Graph kernel network for partial differential equations
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar · 2003
Earlier work this paper cites.
Direct numerical simulation of homogeneous turbulence with hyperviscosity
A. G. Lamorgese, D. A. Caughey, and S. B. Pope · 2004
Earlier work this paper cites.
Linformer: Self-attention with linear complexity
Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma · 2006
Earlier work this paper cites.
Random features for large-scale kernel machines
Ali Rahimi and Benjamin Recht · 2007
Earlier work this paper cites.
Tensor decompositions and applications
Tamara G. Kolda and Brett W. Bader · 2009
Earlier work this paper cites.
Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions, 2010
Nathan Halko, Per-Gunnar Martinsson, and Joel A. Tropp · 2010
Earlier work this paper cites.
Fourier neural operator for parametric partial differential equations
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar · 2010
Earlier work this paper cites.
Tensor-train decomposition
I. V. Oseledets · 2011
Earlier work this paper cites.
Invariant recurrent solutions embedded in a turbulent two-dimensional kolmogorov flow
Gary J. Chandler and Rich R. Kerswell · 2013
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate, 2014
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio · 2014
Earlier work this paper cites.
Alex Graves, Greg Wayne, and Ivo Danihelka · 2014
Earlier work this paper cites.
Speeding-up convolutional neural networks using fine-tuned cp-decomposition, 2015
Vadim Lebedev, Yaroslav Ganin, Maksim Rakhuba, Ivan Oseledets, and Victor Lempitsky · 2015
Earlier work this paper cites.
Effective approaches to attention-based neural machine translation, 2015
Minh-Thang Luong, Hieu Pham, and Christopher D. Manning · 2015
Earlier work this paper cites.
Tensorizing neural networks, 2015
Alexander Novikov, Dmitry Podoprikhin, Anton Osokin, and Dmitry Vetrov · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
High performance python for direct numerical simulations of turbulent flows
Mikael Mortensen and Hans Petter Langtangen · 2016
Earlier work this paper cites.
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al · 2016
Earlier work this paper cites.
Multi-scale context aggregation by dilated convolutions, 2016
Fisher Yu and Vladlen Koltun · 2016
Earlier work this paper cites.
Data-driven synthesis of smoke flows with CNN-based feature descriptors
Mengyu Chu and Nils Thuerey · 2017
Earlier work this paper cites.
Instance normalization: The missing ingredient for fast stylization, 2017
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Tensor-train recurrent neural networks for video classification, 2017
Yinchong Yang, Denis Krompass, and Volker Tresp · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Earlier work this paper cites.
Solving high-dimensional partial differential equations using deep learning
Jiequn Han, Arnulf Jentzen, and Weinan E · 2018
Earlier work this paper cites.
Learning data-driven discretizations for partial differential equations
Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, and Michael P Brenner · 2019
Earlier work this paper cites.
Generating long sequences with sparse transformers, 2019
Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever · 2019
Earlier work this paper cites.
Super-resolution reconstruction of turbulent flows with machine learning
Kai Fukami, Koji Fukagata, and Kunihiko Taira · 2019
Earlier work this paper cites.
Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids, 2019
Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, and Antonio Torralba · 2019
Earlier work this paper cites.
Lu Lu, Pengzhan Jin, and George Em Karniadakis · 2019
Earlier work this paper cites.
A tensorized transformer for language modeling, 2019
Xindian Ma, Peng Zhang, Shuai Zhang, Nan Duan, Yuexian Hou, Dawei Song, and Ming Zhou · 2019
Earlier work this paper cites.
fpinns: Fractional physics-informed neural networks
Guofei Pang, Lu Lu, and George Em Karniadakis · 2019
Earlier work this paper cites.
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
Maziar Raissi, Paris Perdikaris, and George E Karniadakis · 2019
Earlier work this paper cites.
Transformer dissection: A unified understanding of transformer’s attention via the lens of kernel, 2019
Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, and Ruslan Salakhutdinov · 2019
Earlier work this paper cites.
Longformer: The long-document transformer, 2020
Iz Beltagy, Matthew E. Peters, and Arman Cohan · 2020
Cited alongside, same era.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Cited alongside, same era.
End-to-end object detection with transformers, 2020
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko · 2020
Cited alongside, same era.
Deep spatial transformers for autoregressive data-driven forecasting of geophysical turbulence
Ashesh Chattopadhyay, Mustafa Mustafa, Pedram Hassanzadeh, and Karthik Kashinath · 2020
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020
Cited alongside, same era.
Learning green’s functions associated with time-dependent partial differential equations
Nicolas Boullé, Seick Kim, Tianyi Shi, and Alex Townsend · 2022
Later among the works it cites.
How to understand masked autoencoders, 2022
Shuhao Cao, Peng Xu, and David A. Clifton · 2022
Later among the works it cites.
Rethinking attention with performers, 2022
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, and Adrian Weller · 2022
Later among the works it cites.
Learning to correct spectral methods for simulating turbulent flows, 2022
Gideon Dresdner, Dmitrii Kochkov, Peter Norgaard, Leonardo Zepeda-Núñez, Jamie A. Smith, Michael P. Brenner, and Stephan Hoyer · 2022
Later among the works it cites.
Earthformer: Exploring space-time transformers for earth system forecasting
Zhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu, Yuyang Bernie Wang, Mu Li, and Dit-Yan Yeung · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Axial attention in multidimensional transformers, 2020
Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn, and Tim Salimans · 2020
Cited alongside, same era.
Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations
Ameya D Jagtap and George Em Karniadakis · 2020
Cited alongside, same era.
Meshfreeflownet: A physics-constrained deep continuous space-time super-resolution framework
Chiyu “Max” Jiang, Soheil Esmaeilzadeh, Kamyar Azizzadenesheli, Karthik Kashinath, Mustafa Mustafa, Hamdi A. Tchelepi, Philip Marcus, Mr Prabhat, and Anima Anandkumar · 2020
Cited alongside, same era.
Transformers are RNNs: Fast autoregressive transformers with linear attention
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret · 2020
Cited alongside, same era.
Reformer: The efficient transformer
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya · 2020
Cited alongside, same era.
Using machine learning to augment coarse-grid computational fluid dynamics simulations
Jaideep Pathak, Mustafa Mustafa, Karthik Kashinath, Emmanuel Motheau, Thorsten Kurth, and Marcus Day · 2020
Cited alongside, same era.
WeatherBench: A benchmark data set for data-driven weather forecasting
Stephan Rasp, Peter D. Dueben, Sebastian Scher, Jonathan A. Weyn, Soukayna Mouatadid, and Nils Thuerey · 2020
Cited alongside, same era.
Transformers for modeling physical systems
Nicholas Geneva and Nicholas Zabaras · 2022
Later among the works it cites.
Transformer meets boundary value inverse problems
Ruchi Guo, Shuhao Cao, and Long Chen · 2022
Later among the works it cites.
Predicting physics in mesh-reduced space with temporal attention
Xu Han, Han Gao, Tobias Pfaff, Jian-Xun Wang, and Li-Ping Liu · 2022
Later among the works it cites.
Mionet: Learning multiple-input operators via tensor product
Pengzhan Jin, Shuai Meng, and Lu Lu · 2022
Later among the works it cites.
Learning operators with coupled attention, 2022
Georgios Kissas, Jacob Seidman, Leonardo Ferreira Guilhoto, Victor M. Preciado, George J. Pappas, and Paris Perdikaris · 2022
Later among the works it cites.
Graphcast: Learning skillful medium-range global weather forecasting, 2022
Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Alexander Pritzel, Suman Ravuri, Timo Ewalds, Ferran Alet, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Jacklynn Stott, Oriol Vinyals, Shakir Mohamed, and Peter Battaglia · 2022
Later among the works it cites.
Graph neural network-accelerated lagrangian fluid simulation
Zijie Li and Amir Barati Farimani · 2022
Later among the works it cites.
TPU-GAN: Learning temporal coherence from dynamic point cloud sequences
Zijie Li, Tianqin Li, and Amir Barati Farimani · 2022
Later among the works it cites.
Ht-net: Hierarchical transformer based operator learning model for multiscale pdes
Xinliang Liu, Bo Xu, and Lei Zhang · 2022
Later among the works it cites.
Learning the solution operator of boundary value problems using graph neural networks
Winfried Lötzsch, Simon Ohler, and Johannes Otterbach · 2022
Later among the works it cites.
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis · 2022
Later among the works it cites.
Fourierformer: Transformer meets generalized fourier integral theorem
Tan Nguyen, Minh Pham, Tam Nguyen, Khai Nguyen, Stanley Osher, and Nhat Ho · 2022
Later among the works it cites.
Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators, 2022
Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, and Animashree Anandkumar · 2022
Later among the works it cites.
Guaranteed conservation of momentum for learning particle-based fluid dynamics, 2022
Lukas Prantl, Benjamin Ummenhofer, Vladlen Koltun, and Nils Thuerey · 2022
Later among the works it cites.
High-resolution image synthesis with latent diffusion models, 2022
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer · 2022
Later among the works it cites.
Learned coarse models for efficient turbulence simulation, 2022
Kimberly Stachenfeld, Drummond B. Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Sanchez-Gonzalez · 2022
Later among the works it cites.
Roformer: Enhanced transformer with rotary position embedding, 2022
Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu · 2022
Later among the works it cites.
Neural green’s function for laplacian systems
Jingwei Tang, Vinicius C Azevedo, Guillaume Cordonnier, and Barbara Solenthaler · 2022
Later among the works it cites.
Respecting causality is all you need for training physics-informed neural networks, 2022
Sifan Wang, Shyam Sankaran, and Paris Perdikaris · 2022
Later among the works it cites.
U-fno—an enhanced fourier neural operator-based deep-learning model for multiphase flow
Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M Benson · 2022
Later among the works it cites.
Continuous spatiotemporal transformers
Antonio H de O Fonseca, Emanuele Zappala, Josue Ortega Caro, and David van Dijk · 2023
Closest in time.
EAGLE: Large-scale learning of turbulent fluid dynamics with mesh transformers
Steeven JANNY, Aurélien Bénéteau, Madiha Nadri, Julie Digne, Nicolas THOME, and Christian Wolf · 2023
Closest in time.
Multi-grid tensorized fourier neural operator for high resolution PDEs, 2023
Jean Kossaifi, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, and Anima Anandkumar · 2023
Closest in time.
Mitigating spectral bias for the multiscale operator learning with hierarchical attention, 2023
Xinliang Liu, Bo Xu, and Lei Zhang · 2023
Closest in time.
Climax: A foundation model for weather and climate, 2023
Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta, and Aditya Grover · 2023
Closest in time.
Vito: Vision transformer-operator
Oded Ovadia, Adar Kahana, Panos Stinis, Eli Turkel, and George Em Karniadakis · 2023
Closest in time.
U-no: U-shaped neural operators, 2023
Md Ashiqur Rahman, Zachary E. Ross, and Kamyar Azizzadenesheli · 2023
Closest in time.
A physics-informed diffusion model for high-fidelity flow field reconstruction
Dule Shu, Zijie Li, and Amir Barati Farimani · 2023
Closest in time.
Factorized fourier neural operators, 2023
Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong · 2023
Closest in time.
Reliable extrapolation of deep neural operators informed by physics or sparse observations
Min Zhu, Handi Zhang, Anran Jiao, George Em Karniadakis, and Lu Lu · 2023
Closest in time.