Fetching the paper…
Reading the bibliography…
We show that viewing graphs as sets of node features and incorporating structural and positional information into a transformer architecture is able to outperform representations learned with classical graph neural networks (GNNs).
Mutagenicity of nitrofluoranthenes, 3-aminofluoranthene and 1-nitropyrene in Chinese hamster V79 cells
D. L. Berry, G. M. Schoofs, and W. A. Vance · 1985
Earlier work this paper cites.
Mutagenic activity of halogenated propanes and propenes: effect of bromine and chlorine positioning
M. Låg, J. G. Omichinski, E. Dybing, S. D. Nelson, and E. J. SÞderlund · 1994
Earlier work this paper cites.
Effects of the nitro-group on the mutagenicity and toxicity of some benzamines
K.-T. Chung, C. A. Murdock, Y. Zhou, S. E. Stevens Jr., Y.-S. Li, C.-I. Wei, S. Y. Fernando, and M.-W. Chou · 1996
Earlier work this paper cites.
Using the Nyström method to speed up kernel machines
C. K. Williams and M. Seeger · 2001
Earlier work this paper cites.
Laplacian eigenmaps for dimensionality reduction and data representation
M. Belkin and P. Niyogi · 2003
Earlier work this paper cites.
On graph kernels: Hardness results and efficient alternatives
T. Gärtner, P. Flach, and S. Wrobel · 2003
Earlier work this paper cites.
Kernels and regularization on graphs
A. J. Smola and R. Kondor · 2003
Earlier work this paper cites.
Diffusion kernels
R. Kondor and J.-P. Vert · 2004
Earlier work this paper cites.
Shortest-path kernels on graphs
K. Borgwardt and H.-P. Kriegel · 2005
Earlier work this paper cites.
Image classification with segmentation graph kernels
Z. Harchaoui and F. Bach · 2007
Earlier work this paper cites.
The graph neural network model
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini · 2008
Earlier work this paper cites.
Graph kernels based on tree patterns for molecules
P. Mahé and J.-P. Vert · 2009
Earlier work this paper cites.
Efficient graphlet kernels for large graph comparison
N. Shervashidze, S. Vishwanathan, T. Petri, K. Mehlhorn, and K. Borgwardt · 2009
Earlier work this paper cites.
Weisfeiler-Lehman graph kernels
N. Shervashidze, P. Schweitzer, E. J. v. Leeuwen, K. Mehlhorn, and K. M. Borgwardt · 2011
Earlier work this paper cites.
Convolutional networks on graphs for learning molecular fingerprints
D. Duvenaud, D. Maclaurin, J. Aguilera-Iparraguirre, R. Gómez-Bombarelli, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams · 2015
Cited alongside, same era.
Benchmark data sets for graph kernels, 2016
K. Kersting, N. M. Kriege, C. Morris, P. Mutzel, and M. Neumann · 2016
Cited alongside, same era.
A decomposable attention model for natural language inference
A. Parikh, O. Täckström, D. Das, and J. Uszkoreit · 2016
Cited alongside, same era.
Geometric deep learning: Going beyond euclidean data
M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst · 2017
Cited alongside, same era.
Semi-supervised classification with graph convolutional networks
T. N. Kipf and M. Welling · 2017
Cited alongside, same era.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin · 2017
Transformer dissection: A unified understanding of transformer’s attention via the lens of kernel
Y.-H. H. Tsai, S. Bai, M. Yamada, L.-P. Morency, and R. Salakhutdinov · 2019
Later among the works it cites.
How powerful are graph neural networks?
K. Xu, W. Hu, J. Leskovec, and S. Jegelka · 2019
Later among the works it cites.
Graph transformer networks
S. Yun, M. Jeong, R. Kim, J. Kang, and H. Kim · 2019
Later among the works it cites.
Longformer: The long-document transformer
I. Beltagy, M. E. Peters, and A. Cohan · 2020
Later among the works it cites.
Graph kernels: State-of-the-art and future challenges
K. Borgwardt, E. Ghisu, F. Llinares-López, L. O’Bray, and B. Rieck · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Self-attention with relative position representations
P. Shaw, J. Uszkoreit, and A. Vaswani · 2018
Cited alongside, same era.
Graph attention networks
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio · 2018
Cited alongside, same era.
Feastnet: Feature-steered graph convolutions for 3d shape analysis
N. Verma, E. Boyer, and J. Verbeek · 2018
Cited alongside, same era.
Graph neural tangent kernel: Fusing graph neural networks with graph kernels
S. S. Du, K. Hou, R. R. Salakhutdinov, B. Poczos, R. Wang, and K. Xu · 2019
Cited alongside, same era.
Fast graph representation learning with PyTorch Geometric
M. Fey and J. E. Lenssen · 2019
Cited alongside, same era.
Diffusion improves graph learning
J. Klicpera, S. Weißenberger, and S. Günnemann · 2019
Cited alongside, same era.
D. Chen, L. Jacob, and J. Mairal · 2020
Later among the works it cites.
Scaling laws for neural language models, 2020
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei · 2020
Later among the works it cites.
Reformer: The efficient transformer
N. Kitaev, Łukasz Kaiser, and A. Levskaya · 2020
Later among the works it cites.
Improved protein structure prediction using potentials from deep learning
A. W. Senior, R. Evans, J. Jumper, J. Kirkpatrick, L. Sifre, T. Green, et al · 2020
Later among the works it cites.
Efficient transformers: A survey, 2020
Y. Tay, M. Dehghani, D. Bahri, and D. Metzler · 2020
Later among the works it cites.
Graph-bert: Only attention is needed for learning graph representations, 2020
J. Zhang, H. Zhang, C. Xia, and L. Sun · 2020
Later among the works it cites.
An image is worth 16x16 words: Transformers for image recognition at scale
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby · 2021
Closest in time.
A generalization of transformer networks to graphs
V. P. Dwivedi and X. Bresson · 2021
Closest in time.
A trainable optimal transport embedding for feature aggregation and its relationship to attention
G. Mialon, D. Chen, A. d’Aspremont, and J. Mairal · 2021
Closest in time.