Category-Oriented Representation Learning for Image to Multi-Modal Retrieval

Understand

The rise of multi-modal search requests from users has highlighted the importance of multi-modal retrieval (i.e.

• image-to-text or text-to-image retrieval), yet the more complex task of image-to-multi-modal retrieval, crucial for many industry applications, remains under-explored.
• To address this gap and promote further research, we introduce and define the concept of Image-to-Multi-Modal Retrieval (IMMR), a process designed to retrieve rich multi-modal (i.e.
• image and text) documents based on image queries.