Position-aware Self-attention with Relative Positional Encodings for Slot Filling

Understand

This paper describes how to apply self-attention with relative positional encodings to the task of relation extraction.

• We propose to use the self-attention encoder layer together with an additional position-aware attention layer that takes into account positions of the query and the object in the sentence.
• The self-attention encoder also uses a custom implementation of relative positional encodings which allow each word in the sentence to take into account its left and right context.
• The evaluation of the model is done on the TACRED dataset.