Fetching the paper…

Reading the bibliography…

2020

Federated Learning via Synthetic Data

Goetz, Jack, Tewari, Ambuj

Original
PDF
Post on X

Understand

Federated learning allows for the training of a model using data on multiple clients without the clients transmitting that raw data.

However the standard method is to transmit model parameters (or updates), which for modern neural networks can be on the scale of millions of parameters, inflicting significant computational costs on the clients.
We propose a method for federated learning where instead of transmitting a gradient update back to the server, we instead transmit a small amount of synthetic `data'.
We describe the procedure and show some experimental results suggesting this procedure has potential, providing more than an order of magnitude reduction in communication costs with minimal model degradation.

Federated Learning via Synthetic Data · Around

Built on

Soft-label dataset distillation and text dataset distillation
Original
Sucholutsky, I. and Schonlau, M. (2019) · 1910
Earlier work this paper cites.
Advances and open problems in federated learning
Original
Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. (2019) · 1912
Earlier work this paper cites.
Fast exact multiplication by the hessian
Pearlmutter, B. A. (1994) · 1994
Earlier work this paper cites.

Similar

Communication-efficient learning of deep networks from decentralized data
Original
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., et al. (2016) · 2016
Cited alongside, same era.
Towards poisoning of deep learning algorithms with back-gradient optimization
Muñoz-González, L., Biggio, B., Demontis, A., Paudice, A., Wongrassamee, V., Lupu, E. C., and Roli, F. (2017) · 2017
Cited alongside, same era.
Dataset distillation
Original
Wang, T., Zhu, J.-Y., Torralba, A., and Efros, A. A. (2018) · 2018
Cited alongside, same era.

Then

Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., and Smith, V. (2020) · 2020
Closest in time.
A double residual compression algorithm for efficient distributed learning
Liu, X., Li, Y., Tang, J., and Yan, M. (2020) · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…