VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking

Understand

Scale is the primary factor for building a powerful foundation model that could well generalize to a variety of downstream tasks.

• However, it is still challenging to train video foundation models with billions of parameters.
• This paper shows that video masked autoencoder (VideoMAE) is a scalable and general self-supervised pre-trainer for building video foundation models.
• We scale the VideoMAE in both model and data with a core design.