Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization

Understand

The promising zero-shot generalization of vision-language models such as CLIP has led to their adoption using prompt learning for numerous downstream tasks.

• Previous works have shown test-time prompt tuning using entropy minimization to adapt text prompts for unseen domains.
• While effective, this overlooks the key cause for performance degradation to unseen domains -- distribution shift.
• In this work, we explicitly handle this problem by aligning the out-of-distribution (OOD) test sample statistics to those of the source data using prompt tuning.