Contrastive Learning with Negative Sampling Correction

Understand

As one of the most effective self-supervised representation learning methods, contrastive learning (CL) relies on multiple negative pairs to contrast against each positive pair.

• In the standard practice of contrastive learning, data augmentation methods are utilized to generate both positive and negative pairs.
• While existing works have been focusing on improving the positive sampling, the negative sampling process is often overlooked.
• In fact, the generated negative samples are often polluted by positive samples, which leads to a biased loss and performance degradation.