DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding

Understand

Recent studies on open-domain question answering have achieved prominent performance improvement using pre-trained language models such as BERT.

• State-of-the-art approaches typically follow the "retrieve and read" pipeline and employ BERT-based reranker to filter retrieved documents before feeding them into the reader module.
• The BERT retriever takes as input the concatenation of question and each retrieved document.
• Despite the success of these approaches in terms of QA accuracy, due to the concatenation, they can barely handle high-throughput of incoming questions each with a large collection of retrieved documents.