Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment

Understand

Most research on question answering focuses on the pre-deployment stage; i.e., building an accurate model for deployment.

• In this paper, we ask the question: Can we improve QA systems further \emph{post-}deployment based on user interactions? We focus on two kinds of improvements: 1) improving the QA system's performance itself, and 2) providing the model with the ability to explain the correctness or incorrectness of an answer.
• We collect a retrieval-based QA dataset, FeedbackQA, which contains interactive feedback from users.
• We collect this dataset by deploying a base QA system to crowdworkers who then engage with the system and provide feedback on the quality of its answers.