Fetching the paper…
Reading the bibliography…
Attention mechanisms have been widely used to capture long-range dependencies among nodes in Graph Transformers.
Spectral sparsification of graphs
Spielman, D. A. and Teng, S.-H · 2011
Earlier work this paper cites.
A generalization of transformer networks to graphs
Dwivedi, V. and Bresson, X · 2012
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Kipf, T. N. and Welling, M · 2016
Earlier work this paper cites.
Convolutional neural networks on graphs with fast localized spectral filtering, 2017
Defferrard, M., Bresson, X., and Vandergheynst, P · 2017
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks, 2017
Kipf, T. N. and Welling, M · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Residual gated graph convnets, 2018
Bresson, X. and Laurent, T · 2018
Earlier work this paper cites.
Inductive representation learning on large graphs, 2018
Hamilton, W. L., Ying, R., and Leskovec, J · 2018
Earlier work this paper cites.
Graph attention networks, 2018
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y · 2018
Earlier work this paper cites.
How powerful are graph neural networks?
Xu, K., Hu, W., Leskovec, J., and Jegelka, S · 2018
Earlier work this paper cites.
Graph neural networks for social recommendation, 2019
Fan, W., Ma, Y., Li, Q., He, Y., Zhao, E., Tang, J., and Yin, D · 2019
Earlier work this paper cites.
Graph transformer networks
Yun, S., Jeong, M., Kim, R., Kang, J., and Kim, H. J · 2019
Earlier work this paper cites.
Rethinking attention with performers
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al · 2020
Earlier work this paper cites.
O (n) connections are expressive enough: Universal approximability of sparse transformers
Yun, C., Chang, Y.-W., Bhojanapalli, S., Rawat, A. S., Reddi, S., and Kumar, S · 2020
Earlier work this paper cites.
Big bird: Transformers for longer sequences
Zaheer, M., Guruganesh, G., Dubey, K. A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., Yang, L., et al · 2020
Cited alongside, same era.
On the bottleneck of graph neural networks and its practical implications, 2021
Alon, U. and Yahav, E · 2021
Cited alongside, same era.
Convit: Improving vision transformers with soft convolutional inductive biases
D’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., and Sagun, L · 2021
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale, 2021
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N · 2021
Cited alongside, same era.
A generalization of transformer networks to graphs, 2021
Dwivedi, V. P. and Bresson, X · 2021
Cited alongside, same era.
Cmt: Convolutional neural networks meet vision transformers, 2022
Guo, J., Han, K., Wu, H., Tang, Y., Chen, X., Wang, Y., and Xu, C · 2022
Later among the works it cites.
Long range language modeling via gated state spaces
Mehta, H., Gupta, A., Cutkosky, A., and Neyshabur, B · 2022
Later among the works it cites.
Recipe for a general, powerful, scalable graph transformer
Rampášek, L., Galkin, M., Dwivedi, V. P., Luu, A. T., Wolf, G., and Beaini, D · 2022
Later among the works it cites.
Understanding over-squashing and bottlenecks on graphs via curvature, 2022
Topping, J., Giovanni, F. D., Chamberlain, B. P., Dong, X., and Bronstein, M. M · 2022
Later among the works it cites.
Representing long-range context for graph neural networks with global attention, 2022
Wu, Z., Jain, P., Wright, M. A., Mirhoseini, A., Gonzalez, J. E., and Stoica, I · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Freitas, S., Dong, Y., Neil, J., and Chau, D. H · 2021
Cited alongside, same era.
Efficiently modeling long sequences with structured state spaces
Gu, A., Goel, K., and Ré, C · 2021
Cited alongside, same era.
Ammus : A survey of transformer-based pretrained models in natural language processing, 2021
Kalyan, K. S., Rajasekharan, A., and Sangeetha, S · 2021
Cited alongside, same era.
Braingnn: Interpretable brain graph neural network for fmri analysis
Li, X., Zhou, Y., Dvornek, N., Zhang, M., Gao, S., Zhuang, J., Scheinost, D., Staib, L. H., Ventola, P., and Duncan, J. S · 2021
Cited alongside, same era.
Graphit: Encoding graph structure in transformers, 2021
Mialon, G., Chen, D., Selosse, M., and Mairal, J · 2021
Cited alongside, same era.
Do transformers really perform bad for graph representation?, 2021
Ying, C., Cai, T., Luo, S., Zheng, S., Ke, G., He, D., Shen, Y., and Liu, T.-Y · 2021
Cited alongside, same era.
Structure-aware transformer for graph representation learning
Chen, D., O’Bray, L., and Borgwardt, K · 2022
Cited alongside, same era.
Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., and Wang, X · 2022
Later among the works it cites.
Benchmarking graph neural networks
Dwivedi, V. P., Joshi, C. K., Luu, A. T., Laurent, T., Bengio, Y., and Bresson, X · 2023
Later among the works it cites.
Mamba: Linear-time sequence modeling with selective state spaces
Gu, A. and Dao, T · 2023
Later among the works it cites.
Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution
Nguyen, E., Poli, M., Faizi, M., Thomas, A., Birch-Sykes, C., Wornow, M., Patel, A., Rabideau, C., Massaroli, S., Bengio, Y., et al · 2023
Later among the works it cites.
Rwkv: Reinventing rnns for the transformer era
Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Cao, H., Cheng, X., Chung, M., Grella, M., GV, K. K., et al · 2023
Later among the works it cites.
Exphormer: Sparse transformers for graphs
Shirzad, H., Velingker, A., Venkatachalam, B., Sutherland, D. J., and Sinop, A. K · 2023
Later among the works it cites.
Congfu: Conditional graph fusion for drug synergy prediction, 2023
Tsepa, O., Naida, B., Goldenberg, A., and Wang, B · 2023
Later among the works it cites.
Vmamba: Visual state space model
Liu, Y., Tian, Y., Zhao, Y., Yu, H., Xie, L., Wang, Y., Ye, Q., and Liu, Y · 2024
Closest in time.
U-mamba: Enhancing long-range dependency for biomedical image segmentation
Ma, J., Li, F., and Wang, B · 2024
Closest in time.