An Introduction to Transformers

Understand

The transformer is a neural network component that can be used to learn useful representations of sequences or sets of data-points.

• The transformer has driven recent advances in natural language processing, computer vision, and spatio-temporal modelling.
• There are many introductions to transformers, but most do not contain precise mathematical descriptions of the architecture and the intuitions behind the design choices are often also missing.
• Moreover, as research takes a winding path, the explanations for the components of the transformer can be idiosyncratic.