Hierarchical Object-oriented Spatio-Temporal Reasoning for Video Question Answering

Understand

Video Question Answering (Video QA) is a powerful testbed to develop new AI capabilities.

• This task necessitates learning to reason about objects, relations, and events across visual and linguistic domains in space-time.
• High-level reasoning demands lifting from associative visual pattern recognition to symbol-like manipulation over objects, their behavior and interactions.
• Toward reaching this goal we propose an object-oriented reasoning approach in that video is abstracted as a dynamic stream of interacting objects.