2021

Geometric and Physical Quantities Improve E(3) Equivariant Message Passing

Brandstetter, Johannes, Hesselink, Rob, van der Pol, Elise et al.

Understand

Including covariant information, such as position, force, velocity or spin is important in many tasks in computational physics and chemistry.

We introduce Steerable E(3) Equivariant Graph Neural Networks (SEGNNs) that generalise equivariant graph networks, such that node and edge attributes are not restricted to invariant scalars, but can contain covariant information, such as vectors or tensors.
This model, composed of steerable MLPs, is able to incorporate geometric and physical information in both the message and update functions.
Through the definition of steerable node attributes, the MLPs provide a new class of activation functions for general use with steerable feature fields.

Built on

The design and use of steerable filters
William T Freeman, Edward H Adelson, et al · 1991
Earlier work this paper cites.
Canonical decomposition of steerable functions
Yacov. Hel-Or and Patrick Teo · 1996
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
The graph neural network model
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini · 2009
Earlier work this paper cites.
N-body simulations of gravitational dynamics
Walter Dehnen and Justin I Read · 2011
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17
Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean-Louis Reymond · 2012
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Quantum chemistry structures and properties of 134 kilo molecules
Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp, and Anatole von Lilienfeld · 2014
Earlier work this paper cites.
Deep learning
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton · 2015
Earlier work this paper cites.
Deep learning in neural networks: An overview
Jürgen Schmidhuber · 2015
Earlier work this paper cites.
Group equivariant convolutional networks
Taco Cohen and Max Welling · 2016
Earlier work this paper cites.
Convolutional neural networks on graphs with fast localized spectral filtering
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst · 2016
Earlier work this paper cites.
Instance normalization: The missing ingredient for fast stylization
Original
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky · 2016
Earlier work this paper cites.
Steerable cnns
Taco S. Cohen and Max Welling · 2017
Earlier work this paper cites.
Neural message passing for quantum chemistry
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl · 2017
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Thomas N. Kipf and Max Welling · 2017
Earlier work this paper cites.
Schnet: a continuous-filter convolutional neural network for modeling quantum interactions
Schütt Kristof, Kindermans Pieter-Jan, Sauceda Huziel, Chmiela Stefan, Tkatchenko Alexandre, and Klaus-Robert Müller · 2017
Earlier work this paper cites.
Decoupled weight decay regularization
Original
Ilya Loshchilov and Frank Hutter · 2017
Earlier work this paper cites.
Searching for activation functions
Original
Prajit Ramachandran, Barret Zoph, and Quoc V. Le · 2017
Earlier work this paper cites.
Modern Quantum Mechanics
Jun J. Sakurai and Jim Napolitano · 2017
Earlier work this paper cites.
Harmonic networks: Deep translation and rotation equivariance
Daniel E Worrall, Stephan J Garbin, Daniyar Turmukhambetov, and Gabriel J Brostow · 2017
Earlier work this paper cites.

Similar

Relational inductive biases, deep learning, and graph networks
Original
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu · 2018
Cited alongside, same era.
Roto-translation covariant convolutional networks for medical image analysis
Erik J Bekkers, Maxime W Lafarge, Mitko Veta, Koen AJ Eppenhof, Josien PW Pluim, and Remco Duits · 2018
Cited alongside, same era.
Spherical cnns
Taco S. Cohen, Mario Geiger, Jonas Koehler, and Max Welling · 2018
Cited alongside, same era.
Neural relational inference for interacting systems
Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel · 2018
Cited alongside, same era.
On the generalization of equivariance and convolution in neural networks to the action of compact groups
Risi Kondor and Shubhendu Trivedi · 2018
Cited alongside, same era.
Clebsch–gordan nets: a fully fourier space spherical convolutional neural network
Risi Kondor, Zhen Lin, and Shubhendu Trivedi · 2018
Cited alongside, same era.
Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds
Original
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley · 2018
Cited alongside, same era.
3d steerable cnns: Learning rotationally equivariant features in volumetric data
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S Cohen · 2018
Cited alongside, same era.
3d g-cnns for pulmonary nodule detection
Marysia Winkels and Taco S Cohen · 2018
Cited alongside, same era.
Cubenet: Equivariance to 3d rotation and translation
Daniel Worrall and Gabriel Brostow · 2018
Cited alongside, same era.
Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties
Tian Xie and Jeffrey C Grossman · 2018
Cited alongside, same era.
Cormorant: Covariant molecular neural networks
Brandon Anderson, Truong Son Hy, and Risi Kondor · 2019
Cited alongside, same era.

Then

Open catalyst 2020 (oc20) dataset and community challenges
Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Morgane Riviere, Kevin Tran, Javier Heras-Domingo, Caleb Ho, Weihua Hu, Aini Palizhati, Anuroop Sriram, Brandon Wood, Junwoong Yoon, Devi Parikh, C. Lawrence Zitnick, and Zachary Ulissi · 2020
Later among the works it cites.
Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data
Marc Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson · 2020
Later among the works it cites.
Se (3)-transformers: 3d roto-translation equivariant attention networks
Original
Fabian B Fuchs, Daniel E Worrall, Volker Fischer, and Max Welling · 2020
Later among the works it cites.
Learning from protein structure with geometric vector perceptrons
Original
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael JL Townshend, and Ron Dror · 2020
Later among the works it cites.
Fast and uncertainty-aware directional message passing for non-equilibrium molecules
Original
Johannes Klicpera, Shankari Giri, Johannes T Margraf, and Stephan Günnemann · 2020
Later among the works it cites.
A wigner-eckart theorem for group equivariant convolution kernels
Leon Lang and Maurice Weiler · 2020
Later among the works it cites.
Relevance of rotationally equivariant convolutions for predicting molecular properties
Original
Benjamin Kurt Miller, Mario Geiger, Tess E Smidt, and Frank Noé · 2020
Later among the works it cites.
Attentive group equivariant convolutional networks
David Romero, Erik Bekkers, Jakub Tomczak, and Mark Hoogendoorn · 2020
Later among the works it cites.
Learning mesh-based simulation with graph networks, 2020
Pfaff Tobias, Fortunato Meire, Sanchez-Gonzalez Alvaro, and Battaglia Peter W · 2020
Later among the works it cites.
Se (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials
Original
Simon Batzner, Tess E Smidt, Lixin Sun, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, and Boris Kozinsky · 2021
Closest in time.
The open catalyst 2020 (oc20) dataset and community challenges, 2021
Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Morgane Riviere, Kevin Tran, Javier Heras-Domingo, Caleb Ho, Weihua Hu, Aini Palizhati, Anuroop Sriram, Brandon Wood, Junwoong Yoon, Devi Parikh, C. Lawrence Zitnick, and Zachary Ulissi · 2021
Closest in time.
Vector neurons: A general framework for so(3)-equivariant networks
Original
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas Guibas · 2021
Closest in time.
A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups
Original
Marc Finzi, Max Welling, and Andrew Gordon Wilson · 2021
Closest in time.
e3nn/e3nn: 2021-04-21, April 2021a
Mario Geiger, Tess Smidt, Alby M., Benjamin Kurt Miller, Wouter Boomsma, Bradley Dice, Kostiantyn Lapchevskyi, Maurice Weiler, Michał Tyszkiewicz, Simon Batzner, Jes Frellsen, Nuri Jung, Sophia Sanborn, Josh Rackers, and Michael Bailey · 2021
Closest in time.
e3nn/e3nn: 2021-05-10, May 2021b
Mario Geiger, Tess Smidt, Alby M., Benjamin Kurt Miller, Wouter Boomsma, Bradley Dice, Kostiantyn Lapchevskyi, Maurice Weiler, Michał Tyszkiewicz, Simon Batzner, Jes Frellsen, Nuri Jung, Sophia Sanborn, Josh Rackers, and Michael Bailey · 2021
Closest in time.
Gemnet: Universal directional graph neural networks for molecules
Original
Johannes Klicpera, Florian Becker, and Stephan Günnemann · 2021
Closest in time.
Spherical message passing for 3d graph networks
Original
Yi Liu, Limei Wang, Meng Liu, Xuan Zhang, Bora Oztekin, and Shuiwang Ji · 2021
Closest in time.
Boundary graph neural networks for 3d simulations
Original
Andreas Mayr, Sebastian Lehner, Arno Mayrhofer, Christoph Kloss, Sepp Hochreiter, and Johannes Brandstetter · 2021
Closest in time.
E (n) equivariant graph neural networks
Original
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling · 2021
Closest in time.
Equivariant message passing for the prediction of tensorial properties and molecular spectra
Original
Kristof T Schütt, Oliver T Unke, and Michael Gastegger · 2021
Closest in time.
Coordinate independent convolutional networks–isometry and gauge equivariant convolutions on riemannian manifolds
Original
Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…