Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning

Understand

Recently there has been a lot of interest in learning common representations for multiple views of data.

• Typically, such common representations are learned using a parallel corpus between the two views (say, 1M images and their English captions).
• In this work, we address a real-world scenario where no direct parallel data is available between two views of interest (say, $V_1$ and $V_2$) but parallel data is available between each of these views and a pivot view ($V_3$).
• We propose a model for learning a common representation for $V_1$, $V_2$ and $V_3$ using only the parallel data available between $V_1V_3$ and $V_2V_3$.