Fetching the paper…
Reading the bibliography…
We propose a recipe on how to build a general, powerful, scalable (GPS) graph Transformer with linear complexity and state-of-the-art results on a diverse set of benchmarks.
The reduction of a graph to canonical form and the algebra which appears therein
Boris Weisfeiler and Andrei Leman · 1968
Earlier work this paper cites.
Prediction of physicochemical parameters by atomic contributions
Scott A Wildman and Gordon M Crippen · 1999
Earlier work this paper cites.
Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions
Peter Ertl and Ansgar Schuffenhauer · 2009
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Heat kernel estimates for random walks with degenerate weights
Sebastian Andres, Jean-Dominique Deuschel, and Martin Slowik · 2016
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Thomas N Kipf and Max Welling · 2016
Earlier work this paper cites.
Xavier Bresson and Thomas Laurent · 2017
Earlier work this paper cites.
Neural message passing for quantum chemistry
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Graph attention networks
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio · 2018
Earlier work this paper cites.
On the equivalence between graph isomorphism testing and function approximation with gnns
Zhengdao Chen, Lei Chen, Soledad Villar, and Joan Bruna · 2019
Earlier work this paper cites.
Fast graph representation learning with PyTorch Geometric
Matthias Fey and Jan Eric Lenssen · 2019
Earlier work this paper cites.
Strategies for pre-training graph neural networks
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec · 2019
Earlier work this paper cites.
Spectral modification of graphs for improved spectral clustering
Ioannis Koutis and Huong Le · 2019
Earlier work this paper cites.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter · 2019
Earlier work this paper cites.
Provably powerful graph networks
Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, and Yaron Lipman · 2019
Earlier work this paper cites.
Weisfeiler and Leman go neural: Higher-order graph neural networks
Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe · 2019
Earlier work this paper cites.
Relational pooling for graph representations
Ryan Murphy, Balasubramaniam Srinivasan, Vinayak Rao, and Bruno Ribeiro · 2019
Earlier work this paper cites.
How powerful are graph neural networks?
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka · 2019
Earlier work this paper cites.
Longformer: The long-document transformer
Iz Beltagy, Matthew E. Peters, and Arman Cohan · 2020
Earlier work this paper cites.
Principal neighbourhood aggregation for graph nets
Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković · 2020
Earlier work this paper cites.
A generalization of transformer networks to graphs
Vijay Prakash Dwivedi and Xavier Bresson · 2020
Earlier work this paper cites.
Benchmarking graph neural networks
Vijay Prakash Dwivedi, Chaitanya K Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson · 2020
Cited alongside, same era.
Open Graph Benchmark: Datasets for Machine Learning on Graphs
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec · 2020
Cited alongside, same era.
Reformer: The efficient transformer
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya · 2020
Cited alongside, same era.
My body is a cage: the role of morphology in graph-based incompatible control
Vitaly Kurin, Maximilian Igl, Tim Rocktäschel, Wendelin Boehmer, and Shimon Whiteson · 2020
Cited alongside, same era.
Distance encoding: Design provably more powerful neural networks for graph representation learning
Pan Li, Yanbang Wang, Hongwei Wang, and Jure Leskovec · 2020
Cited alongside, same era.
Ammus: A survey of transformer-based pretrained models in natural language processing
Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, and Sivanesan Sangeetha · 2021
Later among the works it cites.
Rethinking graph transformers with spectral attention
Devin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau, and Prudencio Tossou · 2021
Later among the works it cites.
GraphiT: Encoding graph structure in transformers
Grégoire Mialon, Dexiong Chen, Margot Selosse, and Julien Mairal · 2021
Later among the works it cites.
Long range arena: A benchmark for efficient transformers
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler · 2021
Later among the works it cites.
Graph learning with 1d convolutions on random walks
Jan Toenshoff, Martin Ritzert, Hinrikus Wolf, and Martin Grohe · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
What graph neural networks cannot learn: depth vs width
Andreas Loukas · 2020
Cited alongside, same era.
Graph neural networks exponentially lose expressive power for node classification
Kenta Oono and Taiji Suzuki · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu · 2020
Cited alongside, same era.
A survey on the expressive power of graph neural networks
Ryoma Sato · 2020
Cited alongside, same era.
Linformer: Self-attention with linear complexity
Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma · 2020
Cited alongside, same era.
Design space for graph neural networks
Jiaxuan You, Rex Ying, and Jure Leskovec · 2020
Cited alongside, same era.
Big Bird: Transformers for longer sequences
Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontañón, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed · 2020
Cited alongside, same era.
Later among the works it cites.
Understanding over-squashing and bottlenecks on graphs via curvature
Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, and Michael M Bronstein · 2021
Later among the works it cites.
First place solution of KDD Cup 2021 & OGB large-scale challenge graph prediction track
Chengxuan Ying, Mingqi Yang, Shuxin Zheng, Guolin Ke, Shengjie Luo, Tianle Cai, Chenglin Wu, Yuxin Wang, Yanming Shen, and Di He · 2021
Later among the works it cites.
Labeling trick: A theory of using graph neural networks for multi-node representation learning
Muhan Zhang, Pan Li, Yinglong Xia, Kai Wang, and Long Jin · 2021
Later among the works it cites.
Improving graph neural network expressivity via subgraph isomorphism counting
Giorgos Bouritsas, Fabrizio Frasca, Stefanos P Zafeiriou, and Michael Bronstein · 2022
Closest in time.
Structure-aware transformer for graph representation learning
Dexiong Chen, Leslie O’Bray, and Karsten Borgwardt · 2022
Closest in time.
Long range graph benchmark
Vijay Prakash Dwivedi, Ladislav Rampášek, Mikhail Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini · 2022
Closest in time.
Efficiently modeling long sequences with structured state spaces
Albert Gu, Karan Goel, and Christopher Re · 2022
Closest in time.
A survey on vision transformer
Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, et al · 2022
Closest in time.
Global self-attention as a replacement for graph convolution
Md Shamim Hussain, Mohammed J Zaki, and Dharmashankar Subramanian · 2022
Closest in time.
Sign and basis invariant networks for spectral graph representation learning
Derek Lim, Joshua Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, and Stefanie Jegelka · 2022
Closest in time.
Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar · 2022
Closest in time.
GRPE: Relative positional encoding for graph transformer
Wonpyo Park, Woonggi Chang, Donggeon Lee, Juntae Kim, and Seung won Hwang · 2022
Closest in time.
Benchmarking graphormer on large-scale molecular modeling datasets
Yu Shi, Shuxin Zheng, Guolin Ke, Yifei Shen, Jiacheng You, Jiyan He, Shengjie Luo, Chang Liu, Di He, and Tie-Yan Liu · 2022
Closest in time.
Equivariant and stable positional encoding for more powerful graph neural networks
Haorui Wang, Haoteng Yin, Muhan Zhang, and Pan Li · 2022
Closest in time.
Breaking the expression bottleneck of graph neural networks
Mingqi Yang, Renjian Wang, Yanming Shen, Heng Qi, and Baocai Yin · 2022
Closest in time.
From stars to subgraphs: Uplifting any GNN with local structure awareness
Lingxiao Zhao, Wei Jin, Leman Akoglu, and Neil Shah · 2022
Closest in time.