Video Playback Rate Perception for Self-supervisedSpatio-Temporal Representation Learning

Understand

In self-supervised spatio-temporal representation learning, the temporal resolution and long-short term characteristics are not yet fully explored, which limits representation capabilities of learned models.

• In this paper, we propose a novel self-supervised method, referred to as video Playback Rate Perception (PRP), to learn spatio-temporal representation in a simple-yet-effective way.
• PRP roots in a dilated sampling strategy, which produces self-supervision signals about video playback rates for representation model learning.
• PRP is implemented with a feature encoder, a classification module, and a reconstructing decoder, to achieve spatio-temporal semantic retention in a collaborative discrimination-generation manner.