Fetching the paper…

Reading the bibliography…

2018

Repartitioning of the ComplexWebQuestions Dataset

Talmor, Alon, Berant, Jonathan

Original
PDF
Post on X

Understand

Recently, Talmor and Berant (2018) introduced ComplexWebQuestions - a dataset focused on answering complex questions by decomposing them into a sequence of simpler questions and extracting the answer from retrieved web snippets.

In their work the authors used a pre-trained reading comprehension (RC) model (Salant and Berant, 2018) to extract the answer from the web snippets.
In this short note we show that training a RC model directly on the training data of ComplexWebQuestions reveals a leakage from the training set to the test set that allows to obtain unreasonably high performance.
As a solution, we construct a new partitioning of ComplexWebQuestions that does not suffer from this leakage and publicly release it.

Repartitioning of the ComplexWebQuestions Dataset · Around

Built on

Freebase: a collaboratively created graph database for structuring human knowledge
K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. 2008 · 2008
Earlier work this paper cites.
Building a semantic parser overnight
Y. Wang, J. Berant, and P. Liang. 2015 · 2015
Earlier work this paper cites.
SQuAD: 100,000+ questions for machine comprehension of text
P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang. 2016 · 2016
Earlier work this paper cites.

Similar

The value of semantic parse labeling for knowledge base question answering
W. Yih, M. Richardson, C. Meek, M. Chang, and J. Suh. 2016 · 2016
Cited alongside, same era.
Simple and effective multi-paragraph reading comprehension
Original
C. Clark and M. Gardner. 2017 · 2017
Cited alongside, same era.

Then

Contextualized word representations for reading comprehension
S. Salant and J. Berant. 2018 · 2018
Closest in time.
The web as knowledge-base for answering complex questions
A. Talmor and J. Berant. 2018 · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…