Exploring Data Augmentations on Self-/Semi-/Fully- Supervised Pre-trained Models

Understand

Data augmentation has become a standard component of vision pre-trained models to capture the invariance between augmented views.

• In practice, augmentation techniques that mask regions of a sample with zero/mean values or patches from other samples are commonly employed in pre-trained models with self-/semi-/fully-supervised contrastive losses.
• However, the underlying mechanism behind the effectiveness of these augmentation techniques remains poorly explored.
• To investigate the problems, we conduct an empirical study to quantify how data augmentation affects performance.