VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges

R. C. Staudemeyer and E. R. Morris, “Understanding lstm—a tutorial into long short-term memory recurrent neural networks. arxiv 2019,” arXiv preprint arXiv:1909.09586

Original

1909

Earlier work this paper cites.

Z. S. Harris, “Distributional structure,” Word , vol. 10, no. 2-3, pp. 146–162, 1954

1954

Earlier work this paper cites.

A. W. Burks, D. W. Warren, and J. B. Wright, “An analysis of a logical machine using parenthesis-free notation,” Mathematical tables and other aids to computation , vol. 8, no. 46, pp. 53–57, 1954

1954

Earlier work this paper cites.

H. R. Burke, “Raven’s progressive matrices: A review and critical evaluation,” The Journal of Genetic Psychology , vol. 93, no. 2, pp. 199–228, 1958

1958

Earlier work this paper cites.

L. R. Tucker, “Some mathematical notes on three-mode factor analysis,” Psychometrika , vol. 31, no. 3, pp. 279–311, 1966

1966

Earlier work this paper cites.

T. Kasami, “An efficient recognition and syntax-analysis algorithm for context-free languages,” Coordinated Science Laboratory Report no. R-257 , 1966

1966

Earlier work this paper cites.

Z. Wu and M. Palmer, “Verb semantics and lexical selection,” arXiv preprint cmp-lg/9406033 , 1994

1994

Earlier work this paper cites.

Y. LeCun, Y. Bengio et al. , “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks , vol. 3361, no. 10, p. 1995, 1995

1995

Earlier work this paper cites.

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation , vol. 9, no. 8, pp. 1735–1780, 1997

1997

Earlier work this paper cites.

R. Cadene, H. Ben-Younes, M. Cord, and N. Thome, “Murel: Multimodal relational reasoning for visual question answering,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 1989–1998

1998

Earlier work this paper cites.

H. Liu and P. Singh, “Conceptnet—a practical commonsense reasoning tool-kit,” BT technology journal , vol. 22, no. 4, pp. 211–226, 2004

2004

Earlier work this paper cites.

M. Rickert, M. E. Foster, M. Giuliani, T. By, G. Panin, and A. Knoll, “Integrating language, vision and action for human robot dialog systems,” in International Conference on Universal Access in Human-Computer Interaction . Springer, 2007, pp. 987–995

2007

Earlier work this paper cites.

S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives, “Dbpedia: A nucleus for a web of open data,” in The semantic web . Springer, 2007, pp. 722–735

2007

Earlier work this paper cites.

A. Baumann, M. Boltz, J. Ebling, M. Koenig, H. Loos, M. Merkel, W. Niem, J. Warzelhan, and J. Yu, “A review and comparison of measures for automatic video surveillance systems,” EURASIP Journal on Image and Video Processing , vol. 2008, pp. 1–30, 2008

2008

Earlier work this paper cites.

K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor, “Freebase: a collaboratively created graph database for structuring human knowledge,” in Proceedings of the 2008 ACM SIGMOD international conference on Management of data , 2008, pp. 1247–1250

2008

Earlier work this paper cites.

O. Etzioni, M. Banko, S. Soderland, and D. S. Weld, “Open information extraction from the web,” Communications of the ACM , vol. 51, no. 12, pp. 68–74, 2008

2008

Earlier work this paper cites.

P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and L. Bottou, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.” Journal of machine learning research , vol. 11, no. 12, 2010

2010

Earlier work this paper cites.

A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. R. Hruschka, and T. M. Mitchell, “Toward an architecture for never-ending language learning,” in Twenty-Fourth AAAI conference on artificial intelligence , 2010

2010

Earlier work this paper cites.

G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath et al. , “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal processing magazine , vol. 29, no. 6, pp. 82–97, 2012

2012

Earlier work this paper cites.

N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” in European conference on computer vision . Springer, 2012, pp. 746–760

2012

Earlier work this paper cites.

J. Hoffart, F. M. Suchanek, K. Berberich, and G. Weikum, “Yago2: A spatially and temporally enhanced knowledge base from wikipedia,” Artificial Intelligence , vol. 194, pp. 28–61, 2013

2013

Earlier work this paper cites.

M. Malinowski and M. Fritz, “A multi-world approach to question answering about real-world scenes based on uncertain input,” Advances in neural information processing systems , vol. 27, pp. 1682–1690, 2014

2014

Earlier work this paper cites.

T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision . Springer, 2014, pp. 740–755

2014

Earlier work this paper cites.

N. Tandon, G. De Melo, and G. Weikum, “Acquiring comparative commonsense knowledge from the web,” in Twenty-Eighth AAAI Conference on Artificial Intelligence , 2014

2014

Earlier work this paper cites.

N. Tandon, G. De Melo, F. Suchanek, and G. Weikum, “Webchild: Harvesting and organizing commonsense knowledge from the web,” in Proceedings of the 7th ACM international conference on Web search and data mining , 2014, pp. 523–532

2014

Earlier work this paper cites.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

Original

2014

Earlier work this paper cites.

N. Liu, G. Gimel’farb, and P. Delmas, “High-order mgrf models for contrast/offset invariant texture retrieval,” in Proceedings of the 29th International Conference on Image and Vision Computing New Zealand , 2014, pp. 96–101

2014

Earlier work this paper cites.

J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for word representation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , 2014, pp. 1532–1543

2014

Earlier work this paper cites.

D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473 , 2014

Original

2014

Earlier work this paper cites.

J. Andreas and D. Klein, “How much do word embeddings encode about syntax?” in Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , 2014, pp. 822–827

2014

Earlier work this paper cites.

M.-T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” arXiv preprint arXiv:1508.04025 , 2015

Original

2015

Earlier work this paper cites.

J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, “Long-term recurrent convolutional networks for visual recognition and description,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 2625–2634

2015

Earlier work this paper cites.

M. Ren, R. Kiros, and R. Zemel, “Exploring models and data for image question answering,” Advances in neural information processing systems , vol. 28, pp. 2953–2961, 2015

2015

Earlier work this paper cites.

H. Gao, J. Mao, J. Zhou, Z. Huang, L. Wang, and W. Xu, “Are you talking to a machine? dataset and methods for multilingual image question answering,” arXiv preprint arXiv:1505.05612 , 2015

Original

2015

Earlier work this paper cites.

L. Yu, E. Park, A. C. Berg, and T. L. Berg, “Visual madlibs: Fill in the blank image generation and question answering,” arXiv preprint arXiv:1506.00278 , 2015

Original

2015

Earlier work this paper cites.

J. Andreas, M. Rohrbach, T. Darrell, and D. Klein, “Deep compositional question answering with neural module networks. corr abs/1511.02799 (2015),” arXiv preprint arXiv:1511.02799 , 2015

Original

2015

Earlier work this paper cites.

P. Wang, Q. Wu, C. Shen, A. v. d. Hengel, and A. Dick, “Explicit knowledge-based reasoning for visual question answering,” arXiv preprint arXiv:1511.02570 , 2015

Original

2015

Earlier work this paper cites.

S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “Vqa: Visual question answering,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 2425–2433

2015

Earlier work this paper cites.

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 1–9

2015

Earlier work this paper cites.

B. Zhou, Y. Tian, S. Sukhbaatar, A. Szlam, and R. Fergus, “Simple baseline for visual question answering,” arXiv preprint arXiv:1512.02167 , 2015

Original

2015

Earlier work this paper cites.

A. Jiang, F. Wang, F. Porikli, and Y. Li, “Compositional memory for visual question answering,” arXiv preprint arXiv:1511.05676 , 2015

Original

2015

Earlier work this paper cites.

O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and tell: A neural image caption generator,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 3156–3164

2015

Earlier work this paper cites.

M. Ren, R. Kiros, and R. Zemel, “Image question answering: A visual semantic embedding model and a new dataset,” Proc. Advances in Neural Inf. Process. Syst , vol. 1, no. 2, p. 5, 2015

2015

Earlier work this paper cites.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: towards real-time object detection with region proposal networks,” IEEE transactions on pattern analysis and machine intelligence , vol. 39, no. 6, pp. 1137–1149, 2016

2016

Earlier work this paper cites.

R. Bernardi, R. Cakici, D. Elliott, A. Erdem, E. Erdem, N. Ikizler-Cinbis, F. Keller, A. Muscat, and B. Plank, “Automatic description generation from images: A survey of models, datasets, and evaluation measures,” Journal of Artificial Intelligence Research , vol. 55, pp. 409–442, 2016

2016

Earlier work this paper cites.

Y. Zhu, O. Groth, M. Bernstein, and L. Fei-Fei, “Visual7W: Grounded Question Answering in Images,” in IEEE Conference on Computer Vision and Pattern Recognition , 2016

2016

Earlier work this paper cites.

P. Zhang, Y. Goyal, D. Summers-Stay, D. Batra, and D. Parikh, “Yin and yang: Balancing and answering binary visual questions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2016, pp. 5014–5022

2016

Earlier work this paper cites.

Y. Zhu, O. Groth, M. Bernstein, and L. Fei-Fei, “Visual7w: Grounded question answering in images,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 4995–5004

2016

Earlier work this paper cites.

J. Andreas, M. Rohrbach, T. Darrell, and D. Klein, “Neural module networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 39–48

2016

Earlier work this paper cites.

A. Kembhavi, M. Salvato, E. Kolve, M. Seo, H. Hajishirzi, and A. Farhadi, “A diagram is worth a dozen images,” in European conference on computer vision . Springer, 2016, pp. 235–251

2016

Earlier work this paper cites.

H. Noh, P. H. Seo, and B. Han, “Image question answering using convolutional neural network with dynamic parameter prediction,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 30–38

2016

Earlier work this paper cites.

Z. Yang, X. He, J. Gao, L. Deng, and A. Smola, “Stacked attention networks for image question answering,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 21–29

2016

Cited alongside, same era.

H. Xu and K. Saenko, “Ask, attend and answer: Exploring question-guided spatial attention for visual question answering,” in European Conference on Computer Vision . Springer, 2016, pp. 451–466

2016

Cited alongside, same era.

J. Lu, J. Yang, D. Batra, and D. Parikh, “Hierarchical question-image co-attention for visual question answering,” Advances in neural information processing systems , vol. 29, pp. 289–297, 2016

2016

Cited alongside, same era.

A. Fukui, D. H. Park, D. Yang, A. Rohrbach, T. Darrell, and M. Rohrbach, “Multimodal compact bilinear pooling for visual question answering and visual grounding,” arXiv preprint arXiv:1606.01847 , 2016

Original

2016

Cited alongside, same era.

W. Norcliffe-Brown, S. Vafeias, and S. Parisot, “Learning conditioned graph structures for interpretable visual question answering,” Advances in neural information processing systems , vol. 31, 2018

2018

Later among the works it cites.

D.-K. Nguyen and T. Okatani, “Improved fusion of visual and language representations by dense symmetric co-attention for visual question answering,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 6087–6096

2018

Later among the works it cites.

Y. Zhang, J. Hare, and A. Prügel-Bennett, “Learning to count objects in natural images for visual question answering,” arXiv preprint arXiv:1802.05766 , 2018

Original

2018

Later among the works it cites.

Y. Jiang, V. Natarajan, X. Chen, M. Rohrbach, D. Batra, and D. Parikh, “Pythia v0. 1: the winning entry to the vqa challenge 2018,” arXiv preprint arXiv:1807.09956 , 2018

Original

2018

Later among the works it cites.