Understanding Dataset Design Choices for Multi-hop Reasoning

Understand

Learning multi-hop reasoning has been a key challenge for reading comprehension models, leading to the design of datasets that explicitly focus on it.

• Ideally, a model should not be able to perform well on a multi-hop question answering task without doing multi-hop reasoning.
• In this paper, we investigate two recently proposed datasets, WikiHop and HotpotQA.
• First, we explore sentence-factored models for these tasks; by design, these models cannot do multi-hop reasoning, but they are still able to solve a large number of examples in both datasets.