Towards a Unified View on Visual Parameter-Efficient Transfer Learning

Understand

Parameter efficient transfer learning (PETL) aims at making good use of the representation knowledge in the pre-trained large models by fine-tuning a small number of parameters.

• Recently, taking inspiration from the natural language processing (NLP) domain, popular PETL techniques such as prompt-tuning and Adapter have also been successfully applied to the vision domain.
• However, prefix-tuning remains under-explored for vision tasks.
• In this work, we intend to adapt large vision models (LVMs) to downstream tasks with a good parameter-accuracy trade-off.