SimCSE: Simple Contrastive Learning of Sentence Embeddings

Understand

This paper presents SimCSE, a simple contrastive learning framework that greatly advances state-of-the-art sentence embeddings.

• We first describe an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise.
• This simple method works surprisingly well, performing on par with previous supervised counterparts.
• We find that dropout acts as minimal data augmentation, and removing it leads to a representation collapse.