Built on

Guidelines for drinking-water quality
W. H. Organization · 1993
Earlier work this paper cites.
Indoor air quality and health
A. P. Jones · 1999
Earlier work this paper cites.
Understanding power quality problems
M. H. Bollen · 2000
Earlier work this paper cites.
Food quality and safety: consumer perception and demand
K. G. Grunert · 2005
Earlier work this paper cites.
Dqi: Measuring data quality in nlp
Original
S. Mishra, A. Arunkumar, B. Sachdeva, C. Bryan, and C. Baral · 2005
Earlier work this paper cites.
Our evaluation metric needs an update to encourage generalization
Original
S. Mishra, A. Arunkumar, C. Bryan, and C. Baral · 2007
Earlier work this paper cites.
Unbiased look at dataset bias
A. Torralba and A. A. Efros · 2011
Earlier work this paper cites.
A large annotated corpus for learning natural language inference
Original
S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning · 2015
Earlier work this paper cites.
A corpus and evaluation framework for deeper understanding of commonsense stories
Original
N. Mostafazadeh, N. Chambers, X. He, D. Parikh, D. Batra, L. Vanderwende, P. Kohli, and J. Allen · 2016
Earlier work this paper cites.
The effect of different writing tasks on linguistic style: A case study of the roc story cloze task
Original
R. Schwartz, M. Sap, I. Konstas, L. Zilles, Y. Choi, and N. A. Smith · 2017
Earlier work this paper cites.

Similar

A broad-coverage challenge corpus for sentence understanding through inference
Original
A. Williams, N. Nangia, and S. R. Bowman · 2017
Cited alongside, same era.
Annotation artifacts in natural language inference data
Original
S. Gururangan, S. Swayamdipta, O. Levy, R. Schwartz, S. R. Bowman, and N. A. Smith · 2018
Cited alongside, same era.
How much reading does reading comprehension require? a critical investigation of popular benchmarks
Original
D. Kaushik and Z. C. Lipton · 2018
Cited alongside, same era.
Resound: Towards action recognition without representation bias
Y. Li, Y. Li, and N. Vasconcelos · 2018
Cited alongside, same era.
Hypothesis only baselines in natural language inference
Original
A. Poliak, J. Naradowsky, A. Haldar, R. Rudinger, and B. Van Durme · 2018

Unlearn dataset bias in natural language inference by fitting the residual
Original
H. He, S. Zha, and H. Wang · 2019
Later among the works it cites.
Learning the difference that makes a difference with counterfactually-augmented data
Original
D. Kaushik, E. Hovy, and Z. C. Lipton · 2019
Later among the works it cites.
Repair: Removing representation bias by dataset resampling
Y. Li and N. Vasconcelos · 2019
Later among the works it cites.
Inoculation by fine-tuning: A method for analyzing challenge datasets
N. F. Liu, R. Schwartz, and N. A. Smith · 2019
Later among the works it cites.
simple but effective techniques to reduce biases
Original
R. K. Mahabadi and J. Henderson · 2019
Later among the works it cites.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…

2020

Mishra, Swaroop, Arunkumar, Anjana, Sachdeva, Bhavdeep et al.

A `state of the art' model A surpasses humans in a benchmark B, but fails on similar benchmarks C, D, and E.

What does B have that the other benchmarks do not? Recent research provides the answer: spurious bias.
However, developing A to solve benchmarks B through E does not guarantee that it will solve future benchmarks.
To progress towards a model that `truly learns' an underlying task, we need to quantify the differences between successive benchmarks, as opposed to existing binary and black-box approaches.

Guidelines for drinking-water quality
W. H. Organization · 1993
Earlier work this paper cites.
Indoor air quality and health
A. P. Jones · 1999
Earlier work this paper cites.
Understanding power quality problems
M. H. Bollen · 2000
Earlier work this paper cites.
Food quality and safety: consumer perception and demand
K. G. Grunert · 2005
Earlier work this paper cites.
Dqi: Measuring data quality in nlp
Original
S. Mishra, A. Arunkumar, B. Sachdeva, C. Bryan, and C. Baral · 2005
Earlier work this paper cites.
Our evaluation metric needs an update to encourage generalization
Original
S. Mishra, A. Arunkumar, C. Bryan, and C. Baral · 2007
Earlier work this paper cites.
Unbiased look at dataset bias
A. Torralba and A. A. Efros · 2011
Earlier work this paper cites.
A large annotated corpus for learning natural language inference
Original
S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning · 2015
Earlier work this paper cites.
A corpus and evaluation framework for deeper understanding of commonsense stories
Original
N. Mostafazadeh, N. Chambers, X. He, D. Parikh, D. Batra, L. Vanderwende, P. Kohli, and J. Allen · 2016
Earlier work this paper cites.
The effect of different writing tasks on linguistic style: A case study of the roc story cloze task
Original
R. Schwartz, M. Sap, I. Konstas, L. Zilles, Y. Choi, and N. A. Smith · 2017
Earlier work this paper cites.

A broad-coverage challenge corpus for sentence understanding through inference
Original
A. Williams, N. Nangia, and S. R. Bowman · 2017
Cited alongside, same era.
Annotation artifacts in natural language inference data
Original
S. Gururangan, S. Swayamdipta, O. Levy, R. Schwartz, S. R. Bowman, and N. A. Smith · 2018
Cited alongside, same era.
How much reading does reading comprehension require? a critical investigation of popular benchmarks
Original
D. Kaushik and Z. C. Lipton · 2018
Cited alongside, same era.
Resound: Towards action recognition without representation bias
Y. Li, Y. Li, and N. Vasconcelos · 2018
Cited alongside, same era.
Hypothesis only baselines in natural language inference
Original
A. Poliak, J. Naradowsky, A. Haldar, R. Rudinger, and B. Van Durme · 2018

Unlearn dataset bias in natural language inference by fitting the residual
Original
H. He, S. Zha, and H. Wang · 2019
Later among the works it cites.
Learning the difference that makes a difference with counterfactually-augmented data
Original
D. Kaushik, E. Hovy, and Z. C. Lipton · 2019
Later among the works it cites.
Repair: Removing representation bias by dataset resampling
Y. Li and N. Vasconcelos · 2019
Later among the works it cites.
Inoculation by fine-tuning: A method for analyzing challenge datasets
N. F. Liu, R. Schwartz, and N. A. Smith · 2019
Later among the works it cites.
simple but effective techniques to reduce biases
Original
R. K. Mahabadi and J. Henderson · 2019
Later among the works it cites.