Deciphering the Projection Head: Representation Evaluation Self-supervised Learning

Understand

Self-supervised learning (SSL) aims to learn intrinsic features without labels.

• Despite the diverse architectures of SSL methods, the projection head always plays an important role in improving the performance of the downstream task.
• In this work, we systematically investigate the role of the projection head in SSL.
• Specifically, the projection head targets the uniformity part of SSL, which pushes the dissimilar samples away from each other, thus enabling the encoder to focus on extracting semantic features.