SpaceNLI: Evaluating the Consistency of Predicting Inferences in Space

Understand

While many natural language inference (NLI) datasets target certain semantic phenomena, e.g., negation, tense & aspect, monotonicity, and presupposition, to the best of our knowledge, there is no NLI dataset that involves diverse types of spatial expressions and reasoning.

• We fill this gap by semi-automatically creating an NLI dataset for spatial reasoning, called SpaceNLI.
• The data samples are automatically generated from a curated set of reasoning patterns, where the patterns are annotated with inference labels by experts.
• We test several SOTA NLI systems on SpaceNLI to gauge the complexity of the dataset and the system's capacity for spatial reasoning.