Stack-Captioning: Coarse-to-Fine Learning for Image Captioning

Understand

The existing image captioning approaches typically train a one-stage sentence decoder, which is difficult to generate rich fine-grained descriptions.

• On the other hand, multi-stage image caption model is hard to train due to the vanishing gradient problem.
• In this paper, we propose a coarse-to-fine multi-stage prediction framework for image captioning, composed of multiple decoders each of which operates on the output of the previous stage, producing increasingly refined image descriptions.
• Our proposed learning approach addresses the difficulty of vanishing gradients during training by providing a learning objective function that enforces intermediate supervisions.