Transformers in Action Recognition: A Review on Temporal Modeling

Understand

In vision-based action recognition, spatio-temporal features from different modalities are used for recognizing activities.

• Temporal modeling is a long challenge of action recognition.
• However, there are limited methods such as pre-computed motion features, three-dimensional (3D) filters, and recurrent neural networks (RNN) for modeling motion information in deep-based approaches.
• Recently, transformers success in modeling long-range dependencies in natural language processing (NLP) tasks has gotten great attention from other domains; including speech, image, and video, to rely entirely on self-attention without using sequence-aligned RNNs or convolutions.