Leveraging Auxiliary Text for Deep Recognition of Unseen Visual Relationships

Understand

One of the most difficult tasks in scene understanding is recognizing interactions between objects in an image.

• This task is often called visual relationship detection (VRD).
• We consider the question of whether, given auxiliary textual data in addition to the standard visual data used for training VRD models, VRD performance can be improved.
• We present a new deep model that can leverage additional textual data.