Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives

Understand

Large language models (LLMs) have shown increasing power on various natural language processing (NLP) tasks.

• However, tuning these models for downstream tasks usually needs exorbitant costs or is unavailable due to commercial considerations.
• Recently, black-box tuning has been proposed to address this problem by optimizing task-specific prompts without accessing the gradients and hidden representations.
• However, most existing works have yet fully exploited the potential of gradient-free optimization under the scenario of few-shot learning.