FewFedWeight: Few-shot Federated Learning Framework across Multiple NLP Tasks

Understand

Massively multi-task learning with large language models has recently made substantial progress on few-shot generalization.

• However, this is usually performed in a centralized learning fashion, ignoring the privacy sensitivity issue of (annotated) data used in multiple tasks.
• To mitigate this issue, we propose FewFedWeight, a few-shot federated learning framework across multiple tasks, to achieve the best of both worlds: privacy preservation and cross-task generalization.
• FewFedWeight trains client models in isolated devices without sharing data.