Detailed comparison of communication efficiency of split learning and federated learning

Understand

We compare communication efficiencies of two compelling distributed machine learning approaches of split learning and federated learning.

• We show useful settings under which each method outperforms the other in terms of communication efficiency.
• We consider various practical scenarios of distributed learning setup and juxtapose the two methods under various real-life scenarios.
• We consider settings of small and large number of clients as well as small models (1M - 6M parameters), large models (10M - 200M parameters) and very large models (1 Billion-100 Billion parameters).