Improving Generation and Evaluation of Visual Stories via Semantic Consistency

Understand

Story visualization is an under-explored task that falls at the intersection of many important research directions in both computer vision and natural language processing.

• In this task, given a series of natural language captions which compose a story, an agent must generate a sequence of images that correspond to the captions.
• Prior work has introduced recurrent generative models which outperform text-to-image synthesis models on this task.
• However, there is room for improvement of generated images in terms of visual quality, coherence and relevance.