2015

Compositional Memory for Visual Question Answering

Jiang, Aiwen, Wang, Fang, Porikli, Fatih et al.

Understand

Visual Question Answering (VQA) emerges as one of the most fascinating topics in computer vision recently.

Many state of the art methods naively use holistic visual features with language features into a Long Short-Term Memory (LSTM) module, neglecting the sophisticated interaction between them.
This coarse modeling also blocks the possibilities of exploring finer-grained local features that contribute to the question answering dynamically over time.
This paper addresses this fundamental problem by directly modeling the temporal dynamics between language and all possible local image patches.

E. Tulving, “Episodic and semantic memory,”
1972
Earlier work this paper cites.
Z. Wu and M. Palmer, “Verbs semantics and lexical selection,” in
1994
Earlier work this paper cites.
Bradford Books, MIT Press, 1995
N. Cohen and H. Eichenbaum, · 1995
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,”
1997
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. Hinton, “Imagenet classification with deep convolutional neural networks,” in
2012
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in
2012
Earlier work this paper cites.
A. Graves, A. Mohamed, and G. E. Hinton, “Speech recognition with deep recurrent neural networks,”
Original
2013
Earlier work this paper cites.

C. L. Zitnick and P. Dollár, “Edge boxes: Locating object proposals from edges,” in
2014
Cited alongside, same era.
M. Malinowski and M. Fritz, “A multi-world approach to question answering about real-world scenes based on uncertain input,” in
2014
Cited alongside, same era.
D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,”
Original
2014
Cited alongside, same era.
J. Chung, Ç. Gülçehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” in
2014
Cited alongside, same era.
M. Malinowski, M. Rohrbach, and M. Fritz, “Ask your neurons: A neural-based approach to answering questions about images,” in
2015
Cited alongside, same era.
M. Ren, R. Kiros, and R. Zemel, “Exploring models and data for image question answering,” in
2015
Cited alongside, same era.
J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, “Long-term recurrent convolutional networks for visual recognition and description,” in
2015
Cited alongside, same era.

A. Karpathy and L. Fei-Fei, “Deep visual-semantic alignments for generating image descriptions,” in
2015
Closest in time.
A. Kumar, O. Irsoy, J. Su, J. Bradbury, R. English, B. Pierce, P. Ondruska, I. Gulrajani, and R. Socher, “Ask me anything: Dynamic memory networks for natural language processing,”
Original
2015
Closest in time.
K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y. Bengio, “Show, attend and tell: Neural image caption generation with visual attention,” in
2015
Closest in time.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” in
2015
Closest in time.
S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “VQA: visual question answering,”
Original
2015
Closest in time.
H. Gao, J. Mao, J. Zhou, Z. Huang, L. Wang, and W. Xu, “Are you talking to a machine? dataset and methods for multilingual image question answering,”
Original
2015
Closest in time.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in
2015
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…