BYOL works even without batch statistics

Understand

Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach for image representation.

• From an augmented view of an image, BYOL trains an online network to predict a target network representation of a different augmented view of the same image.
• Unlike contrastive methods, BYOL does not explicitly use a repulsion term built from negative pairs in its training objective.
• Yet, it avoids collapse to a trivial, constant representation.