Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks

Understand

Self-supervised learning is a powerful paradigm for representation learning on unlabelled images.

• A wealth of effective new methods based on instance matching rely on data-augmentation to drive learning, and these have reached a rough agreement on an augmentation scheme that optimises popular recognition benchmarks.
• However, there is strong reason to suspect that different tasks in computer vision require features to encode different (in)variances, and therefore likely require different augmentation strategies.
• In this paper, we measure the invariances learned by contrastive methods and confirm that they do learn invariance to the augmentations used and further show that this invariance largely transfers to related real-world changes in pose and lighting.