Interactive Fusion of Multi-level Features for Compositional Activity Recognition

Understand

To understand a complex action, multiple sources of information, including appearance, positional, and semantic features, need to be integrated.

• However, these features are difficult to be fused since they often differ significantly in modality and dimensionality.
• In this paper, we present a novel framework that accomplishes this goal by interactive fusion, namely, projecting features across different spaces and guiding it using an auxiliary prediction task.
• Specifically, we implement the framework in three steps, namely, positional-to-appearance feature extraction, semantic feature interaction, and semantic-to-positional prediction.