Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval

Understand

Cross-modal retrieval between visual data and natural language description remains a long-standing challenge in multimedia.

• While recent image-text retrieval methods offer great promise by learning deep representations aligned across modalities, most of these methods are plagued by the issue of training with small-scale datasets covering a limited number of images with ground-truth sentences.
• Moreover, it is extremely expensive to create a larger dataset by annotating millions of images with sentences and may lead to a biased model.
• Inspired by the recent success of webly supervised learning in deep neural networks, we capitalize on readily-available web images with noisy annotations to learn robust image-text joint representation.