The NT-Xent loss upper bound

Understand

Self-supervised learning is a growing paradigm in deep representation learning, showing great generalization capabilities and competitive performance in low-labeled data regimes.

• The SimCLR framework proposes the NT-Xent loss for contrastive representation learning.
• The objective of the loss function is to maximize agreement, similarity, between sampled positive pairs.
• This short paper derives and proposes an upper bound for the loss and average similarity.