Action Sensitivity Learning for Temporal Action Localization

Understand

Temporal action localization (TAL), which involves recognizing and locating action instances, is a challenging task in video understanding.

• Most existing approaches directly predict action classes and regress offsets to boundaries, while overlooking the discrepant importance of each frame.
• In this paper, we propose an Action Sensitivity Learning framework (ASL) to tackle this task, which aims to assess the value of each frame and then leverage the generated action sensitivity to recalibrate the training procedure.
• We first introduce a lightweight Action Sensitivity Evaluator to learn the action sensitivity at the class level and instance level, respectively.