Understanding Spatial Relations through Multiple Modalities

Understand

Recognizing spatial relations and reasoning about them is essential in multiple applications including navigation, direction giving and human-computer interaction in general.

• Spatial relations between objects can either be explicit -- expressed as spatial prepositions, or implicit -- expressed by spatial verbs such as moving, walking, shifting, etc.
• Both these, but implicit relations in particular, require significant common sense understanding.
• In this paper, we introduce the task of inferring implicit and explicit spatial relations between two entities in an image.