Hierarchical LSTM with Adjusted Temporal Attention for Video Captioning

Understand

Recent progress has been made in using attention based encoder-decoder framework for video captioning.

• However, most existing decoders apply the attention mechanism to every generated word including both visual words (e.g., "gun" and "shooting") and non-visual words (e.g.
• "the", "a").
• However, these non-visual words can be easily predicted using natural language model without considering visual signals or attention.