Spatio-Temporal Action Detection with Multi-Object Interaction

Understand

Spatio-temporal action detection in videos requires localizing the action both spatially and temporally in the form of an "action tube".

• Nowadays, most spatio-temporal action detection datasets (e.g.
• UCF101-24, AVA, DALY) are annotated with action tubes that contain a single person performing the action, thus the predominant action detection models simply employ a person detection and tracking pipeline for localization.
• However, when the action is defined as an interaction between multiple objects, such methods may fail since each bounding box in the action tube contains multiple objects instead of one person.