TriCoLo: Trimodal Contrastive Loss for Text to Shape Retrieval

Understand

Text-to-shape retrieval is an increasingly relevant problem with the growth of 3D shape data.

• Recent work on contrastive losses for learning joint embeddings over multimodal data has been successful at tasks such as retrieval and classification.
• Thus far, work on joint representation learning for 3D shapes and text has focused on improving embeddings through modeling of complex attention between representations, or multi-task learning.
• We propose a trimodal learning scheme over text, multi-view images and 3D shape voxels, and show that with large batch contrastive learning we achieve good performance on text-to-shape retrieval without complex attention mechanisms or losses.