IMO: Greedy Layer-Wise Sparse Representation Learning for Out-of-Distribution Text Classification with Pre-trained Models

Understand

Machine learning models have made incredible progress, but they still struggle when applied to examples from unseen domains.

• This study focuses on a specific problem of domain generalization, where a model is trained on one source domain and tested on multiple target domains that are unseen during training.
• We propose IMO: Invariant features Masks for Out-of-Distribution text classification, to achieve OOD generalization by learning invariant features.
• During training, IMO would learn sparse mask layers to remove irrelevant features for prediction, where the remaining features keep invariant.