DeepStory: Video Story QA by Deep Embedded Memory Networks

Understand

Question-answering (QA) on video contents is a significant challenge for achieving human-level intelligence as it involves both vision and language in real-world settings.

• Here we demonstrate the possibility of an AI agent performing video story QA by learning from a large amount of cartoon videos.
• We develop a video-story learning model, i.e.
• Deep Embedded Memory Networks (DEMN), to reconstruct stories from a joint scene-dialogue video stream using a latent embedding space of observed data.