ArGue: Attribute-Guided Prompt Tuning for Vision-Language Models

Understand

Although soft prompt tuning is effective in efficiently adapting Vision-Language (V&L) models for downstream tasks, it shows limitations in dealing with distribution shifts.

• We address this issue with Attribute-Guided Prompt Tuning (ArGue), making three key contributions.
• 1) In contrast to the conventional approach of directly appending soft prompts preceding class names, we align the model with primitive visual attributes generated by Large Language Models (LLMs).
• We posit that a model's ability to express high confidence in these attributes signifies its capacity to discern the correct class rationales.