Fetching the paper…
Reading the bibliography…
Federated Distillation (FD) is a popular novel algorithmic paradigm for Federated Learning, which achieves training performance competitive to prior parameter averaging based methods, while additionally allowing the clients to train different model architectures, by distilling the client predictions on an unlabeled auxiliary set of data into a student model.
On the limited memory BFGS method for large scale optimization
Liu, D. C. and Nocedal, J · 1989
Earlier work this paper cites.
Approximating a function and its derivatives using mse-optimal linear combinations of trained feedforward neural networks
Hashem, S. and Schmeiser, B · 1993
Earlier work this paper cites.
When networks disagree: Ensemble methods for hybrid neural networks
Perrone, M. P. and Cooper, L. N · 1993
Earlier work this paper cites.
Learning with ensembles: How overfitting can be useful
Sollich, P. and Krogh, A · 1995
Earlier work this paper cites.
On combining artificial neural nets
Sharkey, A. J. C · 1996
Earlier work this paper cites.
Dynamically weighted ensemble neural networks for classification
Jiménez, D · 1998
Earlier work this paper cites.
Popular ensemble methods: An empirical study
Opitz, D. W. and Maclin, R · 1999
Earlier work this paper cites.
A brief introduction to boosting
Schapire, R. E · 1999
Earlier work this paper cites.
Model compression
Bucila, C., Caruana, R., and Niculescu-Mizil, A · 2006
Earlier work this paper cites.
Domain adaptation with multiple sources
Mansour, Y., Mohri, M., and Rostamizadeh, A · 2008
Earlier work this paper cites.
Benchmarking semi-supervised federated learning
Zhang, Z., Yao, Z., Yang, Y., Yan, Y., Gonzalez, J. E., and Mahoney, M. W · 2008
Earlier work this paper cites.
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
A theory of learning from different domains
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W · 2010
Earlier work this paper cites.
Federated unsupervised representation learning
Zhang, F., Kuang, K., You, Z., Shen, T., Xiao, J., Zhang, Y., Wu, C., Zhuang, Y., and Li, X · 2010
Earlier work this paper cites.
Differentially private empirical risk minimization
Chaudhuri, K., Monteleoni, C., and Sarwate, A. D · 2011
Earlier work this paper cites.
Communication-efficient federated distillation
Sattler, F., Marban, A., Rischke, R., and Samek, W · 2012
Earlier work this paper cites.
Twenty years of mixture of experts
Yuksel, S. E., Wilson, J. N., and Gader, P. D · 2012
Earlier work this paper cites.
The algorithmic foundations of differential privacy
Dwork, C. and Roth, A · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2014
Earlier work this paper cites.
Mixture of experts: A literature survey
Masoudnia, S. and Ebrahimpour, R · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Hinton, G., Vinyals, O., and Dean, J · 2015
Earlier work this paper cites.
Character-level convolutional networks for text classification
Zhang, X., Zhao, J. J., and LeCun, Y · 2015
Earlier work this paper cites.
Aligning books and movies: Towards story-like visual explanations by watching movies and reading books
Zhu, Y., Kiros, R., Zemel, R. S., Salakhutdinov, R., Urtasun, R., Torralba, A., and Fidler, S · 2015
Cited alongside, same era.
Deep learning with differential privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L · 2016
Cited alongside, same era.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Cited alongside, same era.
Pointer sentinel mixture models
Merity, S., Xiong, C., Bradbury, J., and Socher, R · 2016
Cited alongside, same era.
Communication-efficient learning of deep networks from decentralized data
McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A · 2017
Cited alongside, same era.
Federated multi-task learning
Agnostic federated learning
Mohri, M., Sivek, G., and Suresh, A. T · 2019
Later among the works it cites.
Zero-shot knowledge distillation in deep networks
Nayak, G. K., Mopuri, K. R., Shaj, V., Radhakrishnan, V. B., and Chakraborty, A · 2019
Later among the works it cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I · 2019
Later among the works it cites.
Federated evaluation of on-device personalization
Wang, K., Mathews, R., Kiddon, C., Eichner, H., Beaufays, F., and Ramage, D · 2019
Later among the works it cites.
FedDistill: Making bayesian model ensemble applicable to federated learning
Chen, H.-Y. and Chao, W.-L · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Smith, V., Chiang, C., Sanjabi, M., and Talwalkar, A. S · 2017
Cited alongside, same era.
Learning from multiple teacher networks
You, S., Xu, C., Xu, C., and Tao, D · 2017
Cited alongside, same era.
Large scale distributed neural network training through online distillation
Anil, R., Pereyra, G., Passos, A., Ormandi, R., Dahl, G. E., and Hinton, G. E · 2018
Cited alongside, same era.
Algorithms and theory for multiple-source adaptation
Hoffman, J., Mohri, M., and Zhang, N · 2018
Cited alongside, same era.
Jeong, E., Oh, S., Kim, H., Park, J., Bennis, M., and Kim, S · 2018
Cited alongside, same era.
Scalable private learning with PATE
Papernot, N., Song, S., Mironov, I., Raghunathan, A., Talwar, K., and Erlingsson, Ú · 2018
Cited alongside, same era.
MobileNetV2: Inverted residuals and linear bottlenecks
Sandler, M., Howard, A. G., Zhu, M., Zhmoginov, A., and Chen, L · 2018
Cited alongside, same era.
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. E · 2020
Later among the works it cites.
Itahara, S., Nishio, T., Koda, Y., Morikura, M., and Yamamoto, K · 2020
Later among the works it cites.
Federated semi-supervised learning with inter-client consistency
Jeong, W., Yoon, J., Yang, E., and Hwang, S. J · 2020
Later among the works it cites.
TinyBERT: Distilling BERT for natural language understanding
Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q · 2020
Later among the works it cites.
The multilingual amazon reviews corpus
Keung, P., Lu, Y., Szarvas, G., and Smith, N. A · 2020
Later among the works it cites.
Ensemble distillation for robust model fusion in federated learning
Lin, T., Kong, L., Stich, S. U., and Jaggi, M · 2020
Later among the works it cites.
Three approaches for personalization with applications to federated learning
Mansour, Y., Mohri, M., Ro, J., and Suresh, A. T · 2020
Later among the works it cites.
Adaptive federated optimization
Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečnỳ, J., Kumar, S., and McMahan, H. B · 2020
Later among the works it cites.
Federated knowledge distillation
Seo, H., Park, J., Oh, S., Bennis, M., and Kim, S · 2020
Later among the works it cites.
Federated learning in medicine: Facilitating multi-institutional collaborations without sharing patient data
Sheller, M. J., Edwards, B., Reina, G. A., Martin, J., Pati, S., Kotrotsou, A., Milchenko, M., Xu, W., Marcus, D., Colen, R. R., et al · 2020
Later among the works it cites.
Federated model distillation with noise-free differential privacy
Sun, L. and Lyu, L · 2020
Later among the works it cites.
Understanding contrastive representation learning through alignment and uniformity on the hypersphere
Wang, T. and Isola, P · 2020
Later among the works it cites.
A theoretical perspective on differentially private federated multi-task learning
Wu, H., Chen, C., and Wang, L · 2020
Later among the works it cites.
Distilled one-shot federated learning
Zhou, Y., Pu, G., Ma, X., Li, X., and Wu, D · 2020
Later among the works it cites.
Fedh2l: Federated learning with model and statistical heterogeneity
Li, Y., Zhou, W., Wang, H., Mi, H., and Hospedales, T. M · 2021
Closest in time.
A survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., and Srivastava, G · 2021
Closest in time.