Translating Videos to Natural Language Using Deep Recurrent Neural Networks

Understand

Solving the visual symbol grounding problem has long been a goal of artificial intelligence.

• The field appears to be advancing closer to this goal with recent breakthroughs in deep learning for natural language grounding in static images.
• In this paper, we propose to translate videos directly to sentences using a unified deep neural network with both convolutional and recurrent structure.
• Described video datasets are scarce, and most existing methods have been applied to toy domains with a small vocabulary of possible words.