Federated Word2Vec: Leveraging Federated Learning to Encourage Collaborative Representation Learning

Understand

Large scale contextual representation models have significantly advanced NLP in recent years, understanding the semantics of text to a degree never seen before.

• However, they need to process large amounts of data to achieve high-quality results.
• Joining and accessing all these data from multiple sources can be extremely challenging due to privacy and regulatory reasons.
• Federated Learning can solve these limitations by training models in a distributed fashion, taking advantage of the hardware of the devices that generate the data.