A critical analysis of self-supervision, or what we can learn from a single image

Understand

We look critically at popular self-supervision techniques for learning deep convolutional neural networks without manual labels.

• We show that three different and representative methods, BiGAN, RotNet and DeepCluster, can learn the first few layers of a convolutional network from a single image as well as using millions of images and manual labels, provided that strong data augmentation is used.
• However, for deeper layers the gap with manual supervision cannot be closed even if millions of unlabelled images are used for training.
• We conclude that: (1) the weights of the early layers of deep networks contain limited information about the statistics of natural images, that (2) such low-level statistics can be learned through self-supervision just as well as through strong supervision, and that (3) the low-level statistics can be captured via synthetic transformations instead of using a large image dataset.