HCQA @ Ego4D EgoSchema Challenge 2024

Understand

In this report, we present our champion solution for Ego4D EgoSchema Challenge in CVPR 2024.

• To deeply integrate the powerful egocentric captioning model and question reasoning model, we propose a novel Hierarchical Comprehension scheme for egocentric video Question Answering, named HCQA.
• It consists of three stages: Fine-grained Caption Generation, Context-driven Summarization, and Inference-guided Answering.
• Given a long-form video, HCQA captures local detailed visual information and global summarised visual information via Fine-grained Caption Generation and Context-driven Summarization, respectively.