Representation Flow for Action Recognition

Understand

In this paper, we propose a convolutional layer inspired by optical flow algorithms to learn motion representations.

• Our representation flow layer is a fully-differentiable layer designed to capture the `flow' of any representation channel within a convolutional neural network for action recognition.
• Its parameters for iterative flow optimization are learned in an end-to-end fashion together with the other CNN model parameters, maximizing the action recognition performance.
• Furthermore, we newly introduce the concept of learning `flow of flow' representations by stacking multiple representation flow layers.