Towards Unique and Informative Captioning of Images

Understand

Despite considerable progress, state of the art image captioning models produce generic captions, leaving out important image details.

• Furthermore, these systems may even misrepresent the image in order to produce a simpler caption consisting of common concepts.
• In this paper, we first analyze both modern captioning systems and evaluation metrics through empirical experiments to quantify these phenomena.
• We find that modern captioning systems return higher likelihoods for incorrect distractor sentences compared to ground truth captions, and that evaluation metrics like SPICE can be 'topped' using simple captioning systems relying on object detectors.