Towards Practical Few-shot Federated NLP

Understand

Transformer-based pre-trained models have emerged as the predominant solution for natural language processing (NLP).

• Fine-tuning such pre-trained models for downstream tasks often requires a considerable amount of labeled private data.
• In practice, private data is often distributed across heterogeneous mobile devices and may be prohibited from being uploaded.
• Moreover, well-curated labeled data is often scarce, presenting an additional challenge.