ScoreCL: Augmentation-Adaptive Contrastive Learning via Score-Matching Function

Understand

Self-supervised contrastive learning (CL) has achieved state-of-the-art performance in representation learning by minimizing the distance between positive pairs while maximizing that of negative ones.

• Recently, it has been verified that the model learns better representation with diversely augmented positive pairs because they enable the model to be more view-invariant.
• However, only a few studies on CL have considered the difference between augmented views, and have not gone beyond the hand-crafted findings.
• In this paper, we first observe that the score-matching function can measure how much data has changed from the original through augmentation.