Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment

Understand

The quality of training data is one of the crucial problems when a learning-centered approach is employed.

• This paper proposes a new method to investigate the quality of a large corpus designed for the recognizing textual entailment (RTE) task.
• The proposed method, which is inspired by a statistical hypothesis test, consists of two phases: the first phase is to introduce the predictability of textual entailment labels as a null hypothesis which is extremely unacceptable if a target corpus has no hidden bias, and the second phase is to test the null hypothesis using a Naive Bayes model.
• The experimental result of the Stanford Natural Language Inference (SNLI) corpus does not reject the null hypothesis.