Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA

Latent compositional representations improve systematic generalization in grounded question answering

Original

Ben Bogin, Sanjay Subramanian, Matt Gardner, and Jonathan Berant. 2020 · 2007

Earlier work this paper cites.

Yin and yang: Balancing and answering binary visual questions

Peng Zhang, Yash Goyal, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2016 · 2016

Earlier work this paper cites.

Visual7w: Grounded question answering in images

Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei-Fei. 2016 · 2016

Earlier work this paper cites.

Making the v in vqa matter: Elevating the role of image understanding in visual question answering

Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017 · 2017

Earlier work this paper cites.

Visual genome: Connecting language and vision using crowdsourced dense image annotations

Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. 2017 · 2017

Earlier work this paper cites.

Annotation artifacts in natural language inference data

Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018 · 2018

Earlier work this paper cites.

Compositional attention networks for machine reasoning

Drew Arad Hudson and Christopher D. Manning. 2018 · 2018

Earlier work this paper cites.

Semantically equivalent adversarial rules for debugging NLP models

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018 · 2018

Earlier work this paper cites.

SWAG: A large-scale adversarial dataset for grounded commonsense inference

Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018 · 2018

Cited alongside, same era.

Misleading failures of partial-input baselines

Shi Feng, Eric Wallace, and Jordan Boyd-Graber. 2019 · 2019

Cited alongside, same era.

Are we modeling the task or the annotator? an investigation of annotator bias in natural language understanding datasets

Mor Geva, Yoav Goldberg, and Jonathan Berant. 2019 · 2019

Cited alongside, same era.

GQA: A new dataset for real-world visual reasoning and compositional question answering

Drew A. Hudson and Christopher D. Manning. 2019 · 2019

Cited alongside, same era.

Learning the difference that makes a difference with counterfactually-augmented data

Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. 2019 · 2019

Cited alongside, same era.

LXMERT: Learning cross-modality encoder representations from transformers

Hao Tan and Mohit Bansal. 2019 · 2019

Later among the works it cites.

HellaSwag: Can a machine really finish your sentence?

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 2019

Later among the works it cites.

Evaluating models’ local decision boundaries via contrast sets

Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, and Ben Zhou. 2020 · 2020

Later among the works it cites.

Linguistically-informed transformations (LIT): A method for automatically generating contrast sets

Chuanrong Li, Lin Shengshuo, Zeyu Liu, Xinyi Wu, Xuhui Zhou, and Shane Steinert-Threlkeld. 2020 · 2020

Later among the works it cites.

Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data

Shachar Rosenman, Alon Jacovi, and Yoav Goldberg. 2020 · 2020