Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies

Understand

The large models, as predicted by scaling raw forecasts, have made groundbreaking progress in many fields, particularly in natural language generation tasks, where they have approached or even surpassed human levels.

• However, the unprecedented scale of their parameters brings significant computational and storage costs.
• These large models require substantial computational resources and GPU memory to operate.
• When adapting large models to specific downstream tasks, their massive parameter scale poses a significant challenge in fine-tuning on hardware platforms with limited computational power and GPU memory.