Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding

Understand

Self-supervised pretraining (SSP) has emerged as a popular technique in machine learning, enabling the extraction of meaningful feature representations without labelled data.

• In the realm of computer vision, pretrained vision transformers (ViTs) have played a pivotal role in advancing transfer learning.
• Nonetheless, the escalating cost of finetuning these large models has posed a challenge due to the explosion of model size.
• This study endeavours to evaluate the effectiveness of pure self-supervised learning (SSL) techniques in computer vision tasks, obviating the need for finetuning, with the intention of emulating human-like capabilities in generalisation and recognition of unseen objects.