MASTAF: A Model-Agnostic Spatio-Temporal Attention Fusion Network for Few-shot Video Classification

Understand

We propose MASTAF, a Model-Agnostic Spatio-Temporal Attention Fusion network for few-shot video classification.

• MASTAF takes input from a general video spatial and temporal representation,e.g., using 2D CNN, 3D CNN, and Video Transformer.
• Then, to make the most of such representations, we use self- and cross-attention models to highlight the critical spatio-temporal region to increase the inter-class variations and decrease the intra-class variations.
• Last, MASTAF applies a lightweight fusion network and a nearest neighbor classifier to classify each query video.