2019

RUBi: Reducing Unimodal Biases in Visual Question Answering

Cadene, Remi, Dancette, Corentin, Ben-younes, Hedi et al.

Understand

Visual Question Answering (VQA) is the task of answering questions about an image.

Some VQA models often exploit unimodal biases to provide the correct answer without using the image information.
As a result, they suffer from a huge drop in performance when evaluated on data outside their training set distribution.
This critical issue makes them unsuitable for real-world settings.

Built on

Answer them all! toward universal visual question answering models
Robik Shrestha, Kushal Kafle, and Christopher Kanan · 1906
Earlier work this paper cites.
Unbiased look at dataset bias
Antonio Torralba and Alexei A. Efros · 2011
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Efficient estimation of word representations in vector space
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean · 2013
Earlier work this paper cites.
Reporting bias and knowledge acquisition
Jonathan Gordon and Benjamin Van Durme · 2013
Earlier work this paper cites.
Skip-thought vectors
Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler · 2015
Earlier work this paper cites.
Deep visual-semantic alignments for generating image descriptions
Andrej Karpathy and Li Fei-Fei · 2015
Earlier work this paper cites.
VQA: Visual Question Answering
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh · 2015
Earlier work this paper cites.
Visual relationship detection with language priors
Cewu Lu, Ranjay Krishna, Michael Bernstein, and Li Fei-Fei · 2016
Earlier work this paper cites.
Analyzing the behavior of visual question answering models
Aishwarya Agrawal, Dhruv Batra, and Devi Parikh · 2016
Earlier work this paper cites.
Stacked attention networks for image question answering
Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola · 2016
Earlier work this paper cites.
Seeing through the human reporting bias: Visual classifiers from noisy human-centric labels
Ishan Misra, C Lawrence Zitnick, Margaret Mitchell, and Ross Girshick · 2016
Earlier work this paper cites.
Revisiting visual question answering baselines
Allan Jabri, Armand Joulin, and Laurens Van Der Maaten · 2016
Earlier work this paper cites.
Hierarchical question-image co-attention for visual question answering
Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh · 2016
Earlier work this paper cites.
Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?
Abhishek Das, Harsh Agrawal, C. Lawrence Zitnick, Devi Parikh, and Dhruv Batra · 2016
Earlier work this paper cites.

Similar

Visual Dialog
Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José M.F. Moura, Devi Parikh, and Dhruv Batra · 2017
Cited alongside, same era.
GuessWhat?! Visual object discovery through multi-modal dialogue
Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, and Aaron C. Courville · 2017
Cited alongside, same era.
Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering
Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh · 2017
Cited alongside, same era.
An analysis of visual question answering algorithms
Kushal Kafle and Christopher Kanan · 2017
Cited alongside, same era.
CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning
Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick · 2017
Cited alongside, same era.
Men also like shopping: Reducing gender bias amplification using corpus-level constraints
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang · 2017
Cited alongside, same era.
Mutan: Multimodal tucker fusion for visual question answering
Hedi Ben-Younes, Rémi Cadène, Nicolas Thome, and Matthieu Cord · 2017
Cited alongside, same era.
Don’t just assume; look and answer: Overcoming priors for visual question answering
Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi · 2018
Cited alongside, same era.
Vizwiz grand challenge: Answering visual questions from blind people
Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham · 2018
Cited alongside, same era.
Bottom-up and top-down attention for image captioning and visual question answering
Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang · 2018
Cited alongside, same era.
Explainable neural computation via stack neural module networks
Ronghang Hu, Jacob Andreas, Trevor Darrell, and Kate Saenko · 2018
Cited alongside, same era.
Bilinear attention networks
Jin-Hwa Kim, Jaehyun Jun, and Byoung-Tak Zhang · 2018
Cited alongside, same era.

Then

Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases
Pierre Stock and Moustapha Cisse · 2018
Later among the works it cites.
Right for the Right Reason: Training Agnostic Networks
Sen Jia, Thomas Lansdall-Welfare, and Nello Cristianini · 2018
Later among the works it cites.
Women also snowboard: Overcoming bias in captioning models
Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach · 2018
Later among the works it cites.
Robot learning in homes: Improving generalization and reducing dataset bias
Abhinav Gupta, Adithyavairavan Murali, Dhiraj Prakashchand Gandhi, and Lerrel Pinto · 2018
Later among the works it cites.
Blindfold baselines for embodied qa
Original
Ankesh Anand, Eugene Belilovsky, Kyle Kastner, Hugo Larochelle, and Aaron Courville · 2018
Later among the works it cites.
Object hallucination in image captioning
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko · 2018
Later among the works it cites.
Beyond bilinear: Generalized multi-modal factorized high-order pooling for visual question answering
Zhou Yu, Jun Yu, Chenchao Xiang, Jianping Fan, and Dacheng Tao · 2018
Later among the works it cites.
Gqa: a new dataset for compositional question answering over real-world images
Original
Drew A Hudson and Christopher D Manning · 2019
Closest in time.
From recognition to cognition: Visual commonsense reasoning
Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi · 2019
Closest in time.
Murel: Multimodal Relational Reasoning for Visual Question Answering
Remi Cadene, Hedi Ben-Younes, Nicolas Thome, and Matthieu Cord · 2019
Closest in time.
Block: Bilinear superdiagonal fusion for visual question answering and visual relationship detection
Hedi Ben-Younes, Remi Cadene, Nicolas Thome, and Matthieu Cord · 2019
Closest in time.
Explainable and explicit visual reasoning over scene graphs
Juanzi Li Jiaxin Shi, Hanwang Zhang · 2019
Closest in time.
Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
Gao Peng, Zhengkai Jiang, Haoxuan You, Zhengkai Jiang, Pan Lu, Steven Hoi, Xiaogang Wang, and Hongsheng Li · 2019
Closest in time.
Explicit Bias Discovery in Visual Question Answering Models
Varun Manjunatha, Nirat Saini, and Larry S. Davis · 2019
Closest in time.
Shifting the baseline: Single modality performance on visual navigation & qa
Jesse Thomason, Daniel Gordon, and Yonatan Bisk · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…