DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications

Understand

Machine reading comprehension (MRC) is a crucial task in natural language processing and has achieved remarkable advancements.

• However, most of the neural MRC models are still far from robust and fail to generalize well in real-world applications.
• In order to comprehensively verify the robustness and generalization of MRC models, we introduce a real-world Chinese dataset -- DuReader_robust.
• It is designed to evaluate the MRC models from three aspects: over-sensitivity, over-stability and generalization.