On minimal variations for unsupervised representation learning

Understand

Unsupervised representation learning aims at describing raw data efficiently to solve various downstream tasks.

• It has been approached with many techniques, such as manifold learning, diffusion maps, or more recently self-supervised learning.
• Those techniques are arguably all based on the underlying assumption that target functions, associated with future downstream tasks, have low variations in densely populated regions of the input space.
• Unveiling minimal variations as a guiding principle behind unsupervised representation learning paves the way to better practical guidelines for self-supervised learning algorithms.