Robust Named Entity Recognition with Truecasing Pretraining

Understand

Although modern named entity recognition (NER) systems show impressive performance on standard datasets, they perform poorly when presented with noisy data.

• In particular, capitalization is a strong signal for entities in many languages, and even state of the art models overfit to this feature, with drastically lower performance on uncapitalized text.
• In this work, we address the problem of robustness of NER systems in data with noisy or uncertain casing, using a pretraining objective that predicts casing in text, or a truecaser, leveraging unlabeled data.
• The pretrained truecaser is combined with a standard BiLSTM-CRF model for NER by appending output distributions to character embeddings.