Universal Graph Transformer Self-Attention Networks

Understand

We introduce a transformer-based GNN model, named UGformer, to learn graph representations.

• In particular, we present two UGformer variants, wherein the first variant (publicized in September 2019) is to leverage the transformer on a set of sampled neighbors for each input node, while the second (publicized in May 2021) is to leverage the transformer on all input nodes.
• Experimental results demonstrate that the first UGformer variant achieves state-of-the-art accuracies on benchmark datasets for graph classification in both inductive setting and unsupervised transductive setting; and the second UGformer variant obtains state-of-the-art accuracies for inductive text classification.
• The code is available at: \url{https://github.com/daiquocnguyen/Graph-Transformer}.