Cross-Modal Scene Networks

Understand

People can recognize scenes across many different modalities beyond natural images.

• In this paper, we investigate how to learn cross-modal scene representations that transfer across modalities.
• To study this problem, we introduce a new cross-modal scene dataset.
• While convolutional neural networks can categorize scenes well, they also learn an intermediate representation not aligned across modalities, which is undesirable for cross-modal transfer applications.