Now It Sounds Like You: Learning Personalized Vocabulary On Device

Understand

In recent years, Federated Learning (FL) has shown significant advancements in its ability to perform various natural language processing (NLP) tasks.

• This work focuses on applying personalized FL for on-device language modeling.
• Due to limitations of memory and latency, these models cannot support the complexity of sub-word tokenization or beam search decoding, resulting in the decision to deploy a closed-vocabulary language model.
• However, closed-vocabulary models are unable to handle out-of-vocabulary (OOV) words belonging to specific users.