Understand
On-device machine learning (ML) enables the training process to exploit a massive amount of user-generated private data samples.
- To enjoy this benefit, inter-device communication overhead should be minimized.
- With this end, we propose federated distillation (FD), a distributed model training algorithm whose communication payload size is much smaller than a benchmark scheme, federated learning (FL), particularly when the model size is large.
- Moreover, user-generated data samples are likely to become non-IID across devices, which commonly degrades the performance compared to the case with an IID dataset.
Built on
Distilling the Knowledge in a Neural Network
G. E. Hinton, O. Vinyals, and J. Dean · 2014
Earlier work this paper cites.
Tractable Resource Management with Uplink Decoupled Millimeter-Wave Overlay in Ultra-Dense Cellular Networks
J. Park, S.-L. Kim, and J. Zander · 2016
Earlier work this paper cites.
Similar
On-Device Machine Intelligence. Google AI Blog. Feb. 2017 [Online, Accessed: 2018-11-23]
S. Ravi · 2017
Cited alongside, same era.
Communication-Efficient Learning of Deep Networks from Decentralized Data
H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas · 2017
Cited alongside, same era.
Federated Optimization: Distributed Machine Learning for On-Device Intelligence [Online]
J. Konečný, H. B. McMagan, and D. Ramage
Cited in the paper.
On-Device Federated Learning via Blockchain and its Latency Analysis [Online]
H. Kim, J. Park, M. Bennis, and S.-L. Kim
Cited in the paper.
Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications [Online]
S. Samarakoon, M. Bennis, W. Saad, and M. Debbah
Cited in the paper.
Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge [Online]
T. Nishio and R. Yonetani
Cited in the paper.
Federated Learning with Non-IID Data [Online]
Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra
Cited in the paper.
Large Scale Distributed Neural Network Training Through Online Distillation [Online]
R. Anil, G. Pereyra, A. Passos, R. Ormandi, G. E. Dahl, and G. E. Hinton
Cited in the paper.
Conditional Generative Adversarial Nets [Online]
M. Mirza, and S. Osindero
Cited in the paper.
Then
Differentially Private Federated Learning: A Client Level Perspective Differentially Private Federated Learning: A Client Level Perspective
R. C. Geyer, T. Klein, and M. Nabi · 2017
Later among the works it cites.
Smartphones Will Get Even Smarter With On-Device Machine Learning. IEEE Spectrum. Mar. 2018 [Online, Accessed: 2018-11-23]
M. Bennis · 2018
Closest in time.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…