Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers

Understand

For real-world language applications, detecting an out-of-distribution (OOD) sample is helpful to alert users or reject such unreliable samples.

• However, modern over-parameterized language models often produce overconfident predictions for both in-distribution (ID) and OOD samples.
• In particular, language models suffer from OOD samples with a similar semantic representation to ID samples since these OOD samples lie near the ID manifold.
• A rejection network can be trained with ID and diverse outlier samples to detect test OOD samples, but explicitly collecting auxiliary OOD datasets brings an additional burden for data collection.