MULE: Multimodal Universal Language Embedding

Understand

Existing vision-language methods typically support two languages at a time at most.

• In this paper, we present a modular approach which can easily be incorporated into existing vision-language methods in order to support many languages.
• We accomplish this by learning a single shared Multimodal Universal Language Embedding (MULE) which has been visually-semantically aligned across all languages.
• Then we learn to relate MULE to visual data as if it were a single language.