Soft Label Memorization-Generalization for Natural Language Inference

Understand

Often when multiple labels are obtained for a training example it is assumed that there is an element of noise that must be accounted for.

• It has been shown that this disagreement can be considered signal instead of noise.
• In this work we investigate using soft labels for training data to improve generalization in machine learning models.
• However, using soft labels for training Deep Neural Networks (DNNs) is not practical due to the costs involved in obtaining multiple labels for large data sets.