Efficient Image Representation Learning with Federated Sampled Softmax

Understand

Learning image representations on decentralized data can bring many benefits in cases where data cannot be aggregated across data silos.

• Softmax cross entropy loss is highly effective and commonly used for learning image representations.
• Using a large number of classes has proven to be particularly beneficial for the descriptive power of such representations in centralized learning.
• However, doing so on decentralized data with Federated Learning is not straightforward as the demand on FL clients' computation and communication increases proportionally to the number of classes.