Learning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning

Understand

Although promising results have been achieved in video captioning, existing models are limited to the fixed inventory of activities in the training corpus, and do not generalize to open vocabulary scenarios.

• Here we introduce a novel task, zero-shot video captioning, that aims at describing out-of-domain videos of unseen activities.
• Videos of different activities usually require different captioning strategies in many aspects, i.e.
• word selection, semantic construction, and style expression etc, which poses a great challenge to depict novel activities without paired training data.