Integrating Auxiliary Information in Self-supervised Learning

Understand

This paper presents to integrate the auxiliary information (e.g., additional attributes for data such as the hashtags for Instagram images) in the self-supervised learning process.

• We first observe that the auxiliary information may bring us useful information about data structures: for instance, the Instagram images with the same hashtags can be semantically similar.
• Hence, to leverage the structural information from the auxiliary information, we present to construct data clusters according to the auxiliary information.
• Then, we introduce the Clustering InfoNCE (Cl-InfoNCE) objective that learns similar representations for augmented variants of data from the same cluster and dissimilar representations for data from different clusters.