A Closer Look at Spatiotemporal Convolutions for Action Recognition

Understand

In this paper we discuss several forms of spatiotemporal convolutions for video analysis and study their effects on action recognition.

• Our motivation stems from the observation that 2D CNNs applied to individual frames of the video have remained solid performers in action recognition.
• In this work we empirically demonstrate the accuracy advantages of 3D CNNs over 2D CNNs within the framework of residual learning.
• Furthermore, we show that factorizing the 3D convolutional filters into separate spatial and temporal components yields significantly advantages in accuracy.