Delving into Out-of-Distribution Detection with Vision-Language Representations

Understand

Recognizing out-of-distribution (OOD) samples is critical for machine learning systems deployed in the open world.

• The vast majority of OOD detection methods are driven by a single modality (e.g., either vision or language), leaving the rich information in multi-modal representations untapped.
• Inspired by the recent success of vision-language pre-training, this paper enriches the landscape of OOD detection from a single-modal to a multi-modal regime.
• Particularly, we propose Maximum Concept Matching (MCM), a simple yet effective zero-shot OOD detection method based on aligning visual features with textual concepts.