CompMix: A Benchmark for Heterogeneous Question Answering

Understand

Fact-centric question answering (QA) often requires access to multiple, heterogeneous, information sources.

• By jointly considering several sources like a knowledge base (KB), a text collection, and tables from the web, QA systems can enhance their answer coverage and confidence.
• However, existing QA benchmarks are mostly constructed with a single source of knowledge in mind.
• This limits capabilities of these benchmarks to fairly evaluate QA systems that can tap into more than one information repository.