End-to-End Training of Neural Retrievers for Open-Domain Question Answering

Understand

Recent work on training neural retrievers for open-domain question answering (OpenQA) has employed both supervised and unsupervised approaches.

• However, it remains unclear how unsupervised and supervised methods can be used most effectively for neural retrievers.
• In this work, we systematically study retriever pre-training.
• We first propose an approach of unsupervised pre-training with the Inverse Cloze Task and masked salient spans, followed by supervised finetuning using question-context pairs.