2020

ConjNLI: Natural Language Inference Over Conjunctive Sentences

Saha, Swarnadeep, Nie, Yixin, Bansal, Mohit

Understand

Reasoning about conjuncts in conjunctive sentences is important for a deeper understanding of conjunctions in English and also how their usages and semantics differ from conjunctive and disjunctive boolean logic.

Existing NLI stress tests do not consider non-boolean usages of conjunctions and use templates for testing such model knowledge.
Hence, we introduce ConjNLI, a challenge stress-test for natural language inference over conjunctive sentences, where the premise differs from the hypothesis by conjuncts removed, added, or replaced.
These sentences contain single and multiple instances of coordinating conjunctions ("and", "or", "but", "nor") with quantifiers, negations, and requiring diverse boolean and non-boolean inferences over conjuncts.

Built on

Simple bert models for relation extraction and semantic role labeling
Original
Peng Shi and Jimmy Lin. 2019 · 1904
Earlier work this paper cites.
RoBERTa: A robustly optimized BERT pretraining approach
Original
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b · 1907
Earlier work this paper cites.
Improving natural language inference with a pretrained parser
Original
Deric Pang, Lucy H Lin, and Noah A Smith. 2019 · 1909
Earlier work this paper cites.
Huggingface’s transformers: State-of-the-art natural language processing
Original
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019 · 1910
Earlier work this paper cites.
Coordinating conjunctions in English
Lila R Gleitman. 1965 · 1965
Earlier work this paper cites.
Tom, Dick, and Harry, and all the king’s men
Gerald J Massey. 1976 · 1976
Earlier work this paper cites.
Vague language: Some vague expressions in English
Joanna Mary Channell. 1983 · 1983
Earlier work this paper cites.
Behavior analysis of NLI models: Uncovering the influence of three factors on robustness
Ivan Sanchez, Jeff Mitchell, and Sebastian Riedel. 2018 · 1985
Earlier work this paper cites.
The semantics of non-boolean “and”
Jack Hoeksema. 1988 · 1988
Earlier work this paper cites.
Boolean and non-boolean ‘and’
Manfred Krifka. 1990 · 1990
Earlier work this paper cites.
A simple but useful approach to conjunct identification
Rajeev Agarwal and Lois Boggess. 1992 · 1992
Earlier work this paper cites.
Building a large annotated corpus of English: The Penn Treebank
Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993 · 1993
Earlier work this paper cites.
Pragmatic halos
Peter Lasersohn. 1999 · 1999
Earlier work this paper cites.
Introduction to the CoNLL-2005 shared task: Semantic role labeling
Xavier Carreras and Lluís Màrquez. 2005 · 2005
Earlier work this paper cites.
Open information extraction from the web
Michele Banko, Michael J Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Etzioni. 2007 · 2007
Earlier work this paper cites.
Vagueness and grammar: The semantics of relative and absolute gradable adjectives
Christopher Kennedy. 2007 · 2007
Earlier work this paper cites.

Similar

Scalar implicatures: Pragmatic inferences or grammar?
Chungmin Lee. 2008 · 2008
Cited alongside, same era.
Metalinguistic comparison in an alternative semantics for imprecision
Marcin Morzycki. 2011 · 2011
Cited alongside, same era.
Boolean semantics for natural language , volume 23
Edward L Keenan and Leonard M Faltz. 2012 · 2012
Cited alongside, same era.
A large annotated corpus for learning natural language inference
Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015 · 2015
Cited alongside, same era.
A neural network for coordination boundary prediction
Jessica Ficler and Yoav Goldberg. 2016b · 2016
Cited alongside, same era.
Enhanced LSTM for natural language inference
Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017 · 2017
Cited alongside, same era.
’And’: Conjunction Reduction Redux
Barry Schein. 2017 · 2017
Cited alongside, same era.
Inference is everything: Recasting semantic resources into a unified evaluation framework
Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van Durme. 2017 · 2017
Cited alongside, same era.
Stress-testing neural models of natural language inference with multiply-quantified sentences
Original
Atticus Geiger, Ignacio Cases, Lauri Karttunen, and Christopher Potts. 2018 · 2018
Cited alongside, same era.
Breaking NLI systems with sentences that require simple lexical inferences
Max Glockner, Vered Shwartz, and Yoav Goldberg. 2018 · 2018
Cited alongside, same era.
Annotation artifacts in natural language inference data
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A Smith. 2018 · 2018
Cited alongside, same era.
Stress test evaluation for natural language inference
Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018 · 2018
Cited alongside, same era.

Then

A broad-coverage challenge corpus for sentence understanding through inference
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018 · 2018
Later among the works it cites.
Representation of constituents in neural language models: Coordination phrase as a case study
Aixiu An, Peng Qian, Ethan Wilcox, and Roger Levy. 2019 · 2019
Later among the works it cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Later among the works it cites.
Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference
Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019 · 2019
Later among the works it cites.
Analyzing compositionality-sensitivity of NLI models
Yixin Nie, Yicheng Wang, and Mohit Bansal. 2019 · 2019
Later among the works it cites.
EQUATE: A benchmark evaluation framework for quantitative reasoning in natural language inference
Abhilasha Ravichander, Aakanksha Naik, Carolyn Rose, and Eduard Hovy. 2019 · 2019
Later among the works it cites.
How well do NLI models capture verb veridicality?
Alexis Ross and Ellie Pavlick. 2019 · 2019
Later among the works it cites.
Decomposed local models for coordinate structure parsing
Hiroki Teranishi, Hiroyuki Shindo, and Yuji Matsumoto. 2019 · 2019
Later among the works it cites.
Are natural language inference models imppressive? learning implicature and presupposition
Original
Paloma Jeretic, Alex Warstadt, Suvrat Bhooshan, and Adina Williams. 2020 · 2020
Closest in time.
Learning the difference that makes a difference with counterfactually-augmented data
Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. 2020 · 2020
Closest in time.
Adversarial NLI: A new benchmark for natural language understanding
Original
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020 · 2020
Closest in time.
Probing natural language inference models through semantic fragments
Original
Kyle Richardson, Hai Hu, Lawrence S Moss, and Ashish Sabharwal. 2020 · 2020
Closest in time.
BLiMP: The benchmark of linguistic minimal pairs for english
Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R Bowman. 2020 · 2020
Closest in time.
Do neural models learn systematicity of monotonicity inference in natural language?
Original
Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, and Kentaro Inui. 2020 · 2020
Closest in time.
The curse of performance instability in analysis datasets: Consequences, source, and suggestions
Original
Xiang Zhou, Yixin Nie, Hao Tan, and Mohit Bansal. 2020 · 2020
Closest in time.
Adversarial examples for evaluating reading comprehension systems
Robin Jia and Percy Liang. 2017 · 2031
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…