Fetching the paper…

Reading the bibliography…

2016

Generating captions without looking beyond objects

Heuer, Hendrik, Monz, Christof, Smeulders, Arnold W. M.

Original
PDF
Post on X

Understand

This paper explores new evaluation perspectives for image captioning and introduces a noun translation task that achieves comparative image caption generation performance by translating from a set of nouns to captions.

This implies that in image captioning, all word categories other than nouns can be evoked by a powerful language model without sacrificing performance on n-gram precision.
The paper also investigates lower and upper bounds of how much individual word categories in the captions contribute to the final BLEU score.
A large possible improvement exists for nouns, verbs, and prepositions.

Generating captions without looking beyond objects · Around

Built on

Toutanova, K., Manning, C.D.: Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In: Proceedings of the 2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora: Held in Conjunction with the 38th Annual Meeting of the Association for Computational Linguistics - Volume 13. pp. 63–70. EMNLP ’00, Association for Computational Linguistics, Stroudsburg, PA, USA (2000), http://dx.doi.org/10.3115/1117794.1117802
2000
Earlier work this paper cites.
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: A method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. pp. 311–318. ACL ’02, Association for Computational Linguistics, Stroudsburg, PA, USA (2002), http://dx.doi.org/10.3115/1073083.1073135
2002
Earlier work this paper cites.
Toutanova, K., Klein, D., Manning, C.D., Singer, Y.: Feature-rich part-of-speech tagging with a cyclic dependency network. In: In Proceedings of HLT-NAACL 2003. pp. 252–259 (2003)
2003
Earlier work this paper cites.
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2014), http://arxiv.org/abs/1409.0473
Original
2014
Earlier work this paper cites.

Similar

Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: A neural image caption generator. CoRR abs/1411.4555 (2014), http://arxiv.org/abs/1411.4555
Original
2014
Cited alongside, same era.
Chen, X., Fang, H., Lin, T., Vedantam, R., Gupta, S., Dollár, P., Zitnick, C.L.: Microsoft COCO captions: Data collection and evaluation server. CoRR abs/1504.00325 (2015), http://arxiv.org/abs/1504.00325
Original
2015
Cited alongside, same era.
Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3128–3137 (2015)
2015
Cited alongside, same era.

Then

Wu, Q., Shen, C., van den Hengel, A., Liu, L., Dick, A.R.: Image captioning with an intermediate attributes layer. CoRR abs/1506.01144 (2015), http://arxiv.org/abs/1506.01144
Original
2015
Later among the works it cites.
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A.C., Salakhutdinov, R., Zemel, R.S., Bengio, Y.: Show, attend and tell: Neural image caption generation with visual attention. CoRR abs/1502.03044 (2015), http://arxiv.org/abs/1502.03044
Original
2015
Later among the works it cites.
Yao, L., Ballas, N., Cho, K., Smith, J.R., Bengio, Y.: Trainable performance upper bounds for image and video captioning. CoRR abs/1511.04590 (2015), http://arxiv.org/abs/1511.04590
Original
2015
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…