Entity-Switched Datasets: An Approach to Auditing the In-Domain Robustness of Named Entity Recognition Models

Understand

Named entity recognition systems perform well on standard datasets comprising English news.

• But given the paucity of data, it is difficult to draw conclusions about the robustness of systems with respect to recognizing a diverse set of entities.
• We propose a method for auditing the in-domain robustness of systems, focusing specifically on differences in performance due to the national origin of entities.
• We create entity-switched datasets, in which named entities in the original texts are replaced by plausible named entities of the same type but of different national origin.