SFPrompt: Communication-Efficient Split Federated Fine-Tuning for Large Pre-Trained Models over Resource-Limited Devices

Understand

Large pre-trained models have exhibited remarkable achievements across various domains.

• The substantial training costs associated with these models have led to wide studies of fine-tuning for effectively harnessing their capabilities in solving downstream tasks.
• Yet, conventional fine-tuning approaches become infeasible when the model lacks access to downstream data due to privacy concerns.
• Naively integrating fine-tuning approaches with the emerging federated learning frameworks incurs substantial communication overhead and exerts high demand on local computing resources, making it impractical for common resource-limited devices.