Joint Embedding Self-Supervised Learning in the Kernel Regime

Understand

The fundamental goal of self-supervised learning (SSL) is to produce useful representations of data without access to any labels for classifying the data.

• Modern methods in SSL, which form representations based on known or constructed relationships between samples, have been particularly effective at this task.
• Here, we aim to extend this framework to incorporate algorithms based on kernel methods where embeddings are constructed by linear maps acting on the feature space of a kernel.
• In this kernel regime, we derive methods to find the optimal form of the output representations for contrastive and non-contrastive loss functions.