Improving Position Encoding of Transformers for Multivariate Time Series Classification

Understand

Transformers have demonstrated outstanding performance in many applications of deep learning.

• When applied to time series data, transformers require effective position encoding to capture the ordering of the time series data.
• The efficacy of position encoding in time series analysis is not well-studied and remains controversial, e.g., whether it is better to inject absolute position encoding or relative position encoding, or a combination of them.
• In order to clarify this, we first review existing absolute and relative position encoding methods when applied in time series classification.