Text Transformations in Contrastive Self-Supervised Learning: A Review

Understand

Contrastive self-supervised learning has become a prominent technique in representation learning.

• The main step in these methods is to contrast semantically similar and dissimilar pairs of samples.
• However, in the domain of Natural Language Processing (NLP), the augmentation methods used in creating similar pairs with regard to contrastive learning (CL) assumptions are challenging.
• This is because, even simply modifying a word in the input might change the semantic meaning of the sentence, and hence, would violate the distributional hypothesis.