What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning

Understand

Understanding spatial relations is essential for intelligent agents to act and communicate in the physical world.

• Relative directions are spatial relations that describe the relative positions of target objects with regard to the intrinsic orientation of reference objects.
• Grounding relative directions is more difficult than grounding absolute directions because it not only requires a model to detect objects in the image and to identify spatial relation based on this information, but it also needs to recognize the orientation of objects and integrate this information into the reasoning process.
• We investigate the challenging problem of grounding relative directions with end-to-end neural networks.