Attention in Natural Language Processing

Understand

Attention is an increasingly popular mechanism used in a wide range of neural architectures.

• The mechanism itself has been realized in a variety of formats.
• However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing.
• In this article, we define a unified model for attention architectures in natural language processing, with a focus on those designed to work with vector representations of the textual data.