Fetching the paper…
Reading the bibliography…
Despite their widespread success in various domains, Transformer networks have yet to perform well across datasets in the domain of 3D atomistic graphs such as molecules even when 3D-related inductive biases like translational invariance and rotational equivariance are considered.
Approximation by superpositions of a sigmoidal function
George V. Cybenko · 1989
Earlier work this paper cites.
Multilayer feedforward networks are universal approximators
Kurt Hornik, Maxwell Stinchcombe, and Halbert White · 1989
Earlier work this paper cites.
Approximation capabilities of multilayer feedforward networks
Kurt Hornik · 1991
Earlier work this paper cites.
Group theory
Mildred S Dresselhaus, Gene Dresselhaus, and Ado Jorio · 2007
Earlier work this paper cites.
Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17
Lars Ruddigkeit, Ruud van Deursen, Lorenz C. Blum, and Jean-Louis Reymond · 2012
Earlier work this paper cites.
Quantum chemistry structures and properties of 134 kilo molecules
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole von Lilienfeld · 2014
Earlier work this paper cites.
Dropout: A simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov · 2014
Earlier work this paper cites.
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei · 2015
Earlier work this paper cites.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton · 2016
Earlier work this paper cites.
Group equivariant convolutional networks
Taco Cohen and Max Welling · 2016
Earlier work this paper cites.
Harmonic networks: Deep translation and rotation equivariance
Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, and Gabriel J. Brostow · 2016
Earlier work this paper cites.
Group Theory in a Nutshell for Physicists
A. Zee · 2016
Earlier work this paper cites.
Machine learning of accurate energy-conserving molecular force fields
Stefan Chmiela, Alexandre Tkatchenko, Huziel E. Sauceda, Igor Poltavsky, Kristof T. Schütt, and Klaus-Robert Müller · 2017
Earlier work this paper cites.
Sigmoid-weighted linear units for neural network function approximation in reinforcement learning
Stefan Elfwing, Eiji Uchibe, and Kenji Doya · 2017
Earlier work this paper cites.
Neural message passing for quantum chemistry
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl · 2017
Earlier work this paper cites.
Inductive representation learning on large graphs
William L. Hamilton, Rex Ying, and Jure Leskovec · 2017
Earlier work this paper cites.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam · 2017
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Thomas N. Kipf and Max Welling · 2017
Earlier work this paper cites.
Searching for activation functions
Prajit Ramachandran, Barret Zoph, and Quoc V. Le · 2017
Earlier work this paper cites.
Schnet: A continuous-filter convolutional neural network for modeling quantum interactions
K. T. Schütt, P.-J. Kindermans, H. E. Sauceda, S. Chmiela, A. Tkatchenko, and K.-R. Müller · 2017
Earlier work this paper cites.
Quantum-chemical insights from deep tensor neural networks
Kristof T. Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R. Müller, and Alexandre Tkatchenko · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Towards exact molecular dynamics simulations with machine-learned force fields
Stefan Chmiela, Huziel E. Sauceda, Klaus-Robert Müller, and Alexandre Tkatchenko · 2018
Earlier work this paper cites.
Spherical CNNs
Taco S. Cohen, Mario Geiger, Jonas Köhler, and Max Welling · 2018
Earlier work this paper cites.
Clebsch–gordan nets: a fully fourier space spherical convolutional neural network
Risi Kondor, Zhen Lin, and Shubhendu Trivedi · 2018
Earlier work this paper cites.
Interactive supercomputing on 40,000 cores for machine learning and data analysis
Albert Reuther, Jeremy Kepner, Chansup Byun, Siddharth Samsi, William Arcand, David Bestor, Bill Bergeron, Vijay Gadepally, Michael Houle, Matthew Hubbell, Michael Jones, Anna Klein, Lauren Milechin, Julia Mullen, Andrew Prout, Antonio Rosa, Charles Yee, and Peter Michaleas · 2018
Earlier work this paper cites.
Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds
Nathaniel Thomas, Tess E. Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley · 2018
Earlier work this paper cites.
Graph attention networks
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio · 2018
Earlier work this paper cites.
3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco Cohen · 2018
Cited alongside, same era.
Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties
Tian Xie and Jeffrey C. Grossman · 2018
Cited alongside, same era.
Cormorant: Covariant molecular neural networks
Brandon Anderson, Truong-Son Hy, and Risi Kondor · 2019
Cited alongside, same era.
Relational graph attention networks
Dan Busbridge, Dane Sherburn, Pietro Cavallo, and Nils Y. Hammerla · 2019
Cited alongside, same era.
Supervised community detection with line graph neural networks
Zhengdao Chen, Lisha Li, and Joan Bruna · 2019
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby · 2021
Later among the works it cites.
Ogb-lsc: A large-scale challenge for machine learning on graphs
Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec · 2021
Later among the works it cites.
Learning from protein structure with geometric vector perceptrons
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror · 2021
Later among the works it cites.
Transformers in vision: A survey
Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah · 2021
Later among the works it cites.
How to find your friendly neighborhood: Graph attention design with self-supervision
Dongkwan Kim and Alice Oh · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Graph representation learning via hard and channel-wise attention networks
Hongyang Gao and Shuiwang Ji · 2019
Cited alongside, same era.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala · 2019
Cited alongside, same era.
Deep Learning for the Life Sciences
Bharath Ramsundar, Peter Eastman, Patrick Walters, Vijay Pande, Karl Leswing, and Zhenqin Wu · 2019
Cited alongside, same era.
PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges
Oliver T. Unke and Markus Meuwly · 2019
Cited alongside, same era.
How powerful are graph neural networks?
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka · 2019
Cited alongside, same era.
wav2vec 2.0: A framework for self-supervised learning of speech representations
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli · 2020
Cited alongside, same era.
Gemnet: Universal directional graph neural networks for molecules
Johannes Klicpera, Florian Becker, and Stephan Günnemann · 2021
Later among the works it cites.
Rethinking graph transformers with spectral attention
Devin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau, and Prudencio Tossou · 2021
Later among the works it cites.
E(n) equivariant graph neural networks
Víctor Garcia Satorras, Emiel Hoogeboom, and Max Welling · 2021
Later among the works it cites.
Equivariant message passing for the prediction of tensorial properties and molecular spectra
Kristof T. Schütt, Oliver T. Unke, and Michael Gastegger · 2021
Later among the works it cites.
Rotation invariant graph neural networks using spin convolutions
Muhammed Shuaibi, Adeesh Kolluru, Abhishek Das, Aditya Grover, Anuroop Sriram, Zachary Ulissi, and C. Lawrence Zitnick · 2021
Later among the works it cites.
Finding symmetry breaking order parameters with euclidean neural networks
Tess E. Smidt, Mario Geiger, and Benjamin Kurt Miller · 2021
Later among the works it cites.
ATOM3d: Tasks on molecules in three dimensions
Raphael John Lamarre Townshend, Martin Vögele, Patricia Adriana Suriana, Alexander Derry, Alexander Powers, Yianni Laloudakis, Sidhika Balachandar, Bowen Jing, Brandon M. Anderson, Stephan Eismann, Risi Kondor, Russ Altman, and Ron O. Dror · 2021
Later among the works it cites.
SE(3)-equivariant prediction of molecular wavefunctions and electronic densities
Oliver Thorsten Unke, Mihail Bogojeski, Michael Gastegger, Mario Geiger, Tess Smidt, and Klaus Robert Muller · 2021
Later among the works it cites.
Do transformers really perform badly for graph representation?
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu · 2021
Later among the works it cites.
E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky · 2022
Closest in time.
Geometric and physical quantities improve e(3) equivariant message passing
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik J Bekkers, and Max Welling · 2022
Closest in time.
How attentive are graph attention networks?
Shaked Brody, Uri Alon, and Eran Yahav · 2022
Closest in time.
Neural scaling of deep chemical models
Nathan Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Gomez-Bombarelli, Connor Coley, and Vijay Gadepally · 2022
Closest in time.
How do graph networks generalize to large and diverse molecular systems?
Johannes Gasteiger, Muhammed Shuaibi, Anuroop Sriram, Stephan Günnemann, Zachary Ulissi, C. Lawrence Zitnick, and Abhishek Das · 2022
Closest in time.
e3nn/e3nn: 2022-04-13, April 2022
Mario Geiger, Tess Smidt, Alby M., Benjamin Kurt Miller, Wouter Boomsma, Bradley Dice, Kostiantyn Lapchevskyi, Maurice Weiler, Michał Tyszkiewicz, Simon Batzner, Dylan Madisetti, Martin Uhrin, Jes Frellsen, Nuri Jung, Sophia Sanborn, Mingjian Wen, Josh Rackers, Marcel Rød, and Michael Bailey · 2022
Closest in time.
Simple GNN regularisation for 3d molecular property prediction and beyond
Jonathan Godwin, Michael Schaarschmidt, Alexander L Gaunt, Alvaro Sanchez-Gonzalez, Yulia Rubanova, Petar Veličković, James Kirkpatrick, and Peter Battaglia · 2022
Closest in time.
Equivariant graph attention networks for molecular property prediction
Tuan Le, Frank Noé, and Djork-Arné Clevert · 2022
Closest in time.
Spherical message passing for 3d molecular graphs
Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji · 2022
Closest in time.
Learning local equivariant representations for large-scale atomistic dynamics
Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J. Owen, Mordechai Kornbluth, and Boris Kozinsky · 2022
Closest in time.
Benchmarking graphormer on large-scale molecular modeling datasets
Yu Shi, Shuxin Zheng, Guolin Ke, Yifei Shen, Jiacheng You, Jiyan He, Shengjie Luo, Chang Liu, Di He, and Tie-Yan Liu · 2022
Closest in time.
Towards training billion parameter graph neural networks for atomic simulations
Anuroop Sriram, Abhishek Das, Brandon M Wood, and C. Lawrence Zitnick · 2022
Closest in time.
Equivariant transformers for neural network based molecular potentials
Philipp Thölke and Gianni De Fabritiis · 2022
Closest in time.
A recipe for cracking the quantum scaling limit with machine learned electron densities
Joshua A Rackers, Lucas Tecot, Mario Geiger, and Tess E Smidt · 2023
Closest in time.