Evaluating the Robustness of Neural Language Models to Input Perturbations

Understand

High-performance neural language models have obtained state-of-the-art results on a wide range of Natural Language Processing (NLP) tasks.

• However, results for common benchmark datasets often do not reflect model reliability and robustness when applied to noisy, real-world data.
• In this study, we design and implement various types of character-level and word-level perturbation methods to simulate realistic scenarios in which input texts may be slightly noisy or different from the data distribution on which NLP systems were trained.
• Conducting comprehensive experiments on different NLP tasks, we investigate the ability of high-performance language models such as BERT, XLNet, RoBERTa, and ELMo in handling different types of input perturbations.