HittER: Hierarchical Transformers for Knowledge Graph Embeddings

Understand

This paper examines the challenging problem of learning representations of entities and relations in a complex multi-relational knowledge graph.

• We propose HittER, a Hierarchical Transformer model to jointly learn Entity-relation composition and Relational contextualization based on a source entity's neighborhood.
• Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from outputs of the bottom block.
• We further design a masked entity prediction task to balance information from the relational context and the source entity itself.