A Generalization of Transformer Networks to Graphs

Understand

We propose a generalization of transformer neural network architecture for arbitrary graphs.

• The original transformer was designed for Natural Language Processing (NLP), which operates on fully connected graphs representing all connections between the words in a sequence.
• Such architecture does not leverage the graph connectivity inductive bias, and can perform poorly when the graph topology is important and has not been encoded into the node features.
• We introduce a graph transformer with four new properties compared to the standard model.