STH: Spatio-Temporal Hybrid Convolution for Efficient Action Recognition

Understand

Effective and Efficient spatio-temporal modeling is essential for action recognition.

• Existing methods suffer from the trade-off between model performance and model complexity.
• In this paper, we present a novel Spatio-Temporal Hybrid Convolution Network (denoted as "STH") which simultaneously encodes spatial and temporal video information with a small parameter cost.
• Different from existing works that sequentially or parallelly extract spatial and temporal information with different convolutional layers, we divide the input channels into multiple groups and interleave the spatial and temporal operations in one convolutional layer, which deeply incorporates spatial and temporal clues.