Built on

W. Hoeffding, “Probability inequalities for sums of bounded random variables,” Journal of the American Statistical Association
1963
Earlier work this paper cites.
S. Lloyd, “Least squares quantization in pcm,” IEEE transactions on information theory
1982
Earlier work this paper cites.
Y. LeCun, “The mnist database of handwritten digits,” http://yann. lecun. com/exdb/mnist/
1998
Earlier work this paper cites.
Springer Science & Business Media, 2003
Y. Nesterov, Introductory lectures on convex optimization: A basic course · 2003
Earlier work this paper cites.
A. Krizhevsky, G. Hinton, et al
2009
Earlier work this paper cites.
D. Müllner, “Modern hierarchical, agglomerative clustering algorithms,” arXiv preprint arXiv:1109.2378
Original
2011
Earlier work this paper cites.
J. Bolte, S. Sabach, and M. Teboulle, “Proximal alternating linearized minimization for nonconvex and nonsmooth problems,” Mathematical Programming
2013
Earlier work this paper cites.
A. Krizhevsky, V. Nair, and G. Hinton, “The cifar-10 dataset,” online: http://www. cs. toronto. edu/kriz/cifar. html
2014
Earlier work this paper cites.
J. Konečnỳ, B. McMahan, and D. Ramage, “Federated optimization: Distributed optimization beyond the datacenter,” arXiv preprint arXiv:1511.03575
Original
2015
Earlier work this paper cites.
J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492
Original
2016
Earlier work this paper cites.
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial Intelligence and Statistics
2017
Earlier work this paper cites.
R. C. Geyer, T. Klein, and M. Nabi, “Differentially private federated learning: A client level perspective,” arXiv preprint arXiv:1712.07557
Original
2017
Earlier work this paper cites.
V. Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, “Federated multi-task learning,” Advances in neural information processing systems
2017
Earlier work this paper cites.
G. Cohen, S. Afshar, J. Tapson, and A. Van Schaik, “Emnist: Extending mnist to handwritten letters,” in 2017 International Joint Conference on Neural Networks (IJCNN)
2017
Earlier work this paper cites.
H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747
Original
2017
Earlier work this paper cites.
K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” in proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security
2017
Earlier work this paper cites.
Y. Aono, T. Hayashi, L. Wang, S. Moriai, et al
2017
Earlier work this paper cites.

Similar

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” arXiv preprint arXiv:1812.06127
Original
2018
Cited alongside, same era.
A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese, “Taskonomy: Disentangling task transfer learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition
2018
Cited alongside, same era.
Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” arXiv preprint arXiv:1806.00582
Original
2018
Cited alongside, same era.
L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” arXiv preprint arXiv:1802.03426
Original
2018
Cited alongside, same era.

Then

T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine
2020
Later among the works it cites.
A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach,” in Advances in Neural Information Processing Systems
2020
Later among the works it cites.
A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An efficient framework for clustered federated learning,” in Advances in Neural Information Processing Systems
2020
Later among the works it cites.
Y. Mansour, M. Mohri, J. Ro, and A. T. Suresh, “Three approaches for personalization with applications to federated learning,” arXiv preprint arXiv:2002.10619
Original
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2021

Federated Learning with Taskonomy for Non-IID Data

Jamali-Rad, Hadi, Abdizadeh, Mohammad, Singh, Anuj

Understand

Classical federated learning approaches incur significant performance degradation in the presence of non-IID client data.

A possible direction to address this issue is forming clusters of clients with roughly IID data.
Most solutions following this direction are iterative and relatively slow, also prone to convergence issues in discovering underlying cluster formations.
We introduce federated learning with taskonomy (FLT) that generalizes this direction by learning the task-relatedness between clients for more efficient federated aggregation of heterogeneous data.

Built on

W. Hoeffding, “Probability inequalities for sums of bounded random variables,” Journal of the American Statistical Association
1963
Earlier work this paper cites.
S. Lloyd, “Least squares quantization in pcm,” IEEE transactions on information theory
1982
Earlier work this paper cites.
Y. LeCun, “The mnist database of handwritten digits,” http://yann. lecun. com/exdb/mnist/
1998
Earlier work this paper cites.
Springer Science & Business Media, 2003
Y. Nesterov, Introductory lectures on convex optimization: A basic course · 2003
Earlier work this paper cites.
A. Krizhevsky, G. Hinton, et al
2009
Earlier work this paper cites.
D. Müllner, “Modern hierarchical, agglomerative clustering algorithms,” arXiv preprint arXiv:1109.2378
Original
2011
Earlier work this paper cites.
J. Bolte, S. Sabach, and M. Teboulle, “Proximal alternating linearized minimization for nonconvex and nonsmooth problems,” Mathematical Programming
2013
Earlier work this paper cites.
A. Krizhevsky, V. Nair, and G. Hinton, “The cifar-10 dataset,” online: http://www. cs. toronto. edu/kriz/cifar. html
2014
Earlier work this paper cites.
J. Konečnỳ, B. McMahan, and D. Ramage, “Federated optimization: Distributed optimization beyond the datacenter,” arXiv preprint arXiv:1511.03575
Original
2015
Earlier work this paper cites.
J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492
Original
2016
Earlier work this paper cites.
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial Intelligence and Statistics
2017
Earlier work this paper cites.
R. C. Geyer, T. Klein, and M. Nabi, “Differentially private federated learning: A client level perspective,” arXiv preprint arXiv:1712.07557
Original
2017
Earlier work this paper cites.
V. Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, “Federated multi-task learning,” Advances in neural information processing systems
2017
Earlier work this paper cites.
G. Cohen, S. Afshar, J. Tapson, and A. Van Schaik, “Emnist: Extending mnist to handwritten letters,” in 2017 International Joint Conference on Neural Networks (IJCNN)
2017
Earlier work this paper cites.
H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747
Original
2017
Earlier work this paper cites.
K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” in proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security
2017
Earlier work this paper cites.
Y. Aono, T. Hayashi, L. Wang, S. Moriai, et al
2017
Earlier work this paper cites.

Similar

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” arXiv preprint arXiv:1812.06127
Original
2018
Cited alongside, same era.
A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese, “Taskonomy: Disentangling task transfer learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition
2018
Cited alongside, same era.
Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” arXiv preprint arXiv:1806.00582
Original
2018
Cited alongside, same era.
L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” arXiv preprint arXiv:1802.03426
Original
2018
Cited alongside, same era.

Then

T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine
2020
Later among the works it cites.
A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach,” in Advances in Neural Information Processing Systems
2020
Later among the works it cites.
A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An efficient framework for clustered federated learning,” in Advances in Neural Information Processing Systems
2020
Later among the works it cites.
Y. Mansour, M. Mohri, J. Ro, and A. T. Suresh, “Three approaches for personalization with applications to federated learning,” arXiv preprint arXiv:2002.10619
Original
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

J. Wang and G. Joshi, “Cooperative sgd: A unified framework for the design and analysis of communication-efficient sgd algorithms,” arXiv preprint arXiv:1808.07576

Original

2018

Cited alongside, same era.

B. E. Woodworth, J. Wang, A. Smith, B. McMahan, and N. Srebro, “Graph oracle models, lower bounds, and gaps for parallel stochastic optimization,” Advances in neural information processing systems

2018

Cited alongside, same era.

T. Hashimoto, M. Srivastava, H. Namkoong, and P. Liang, “Fairness without demographics in repeated loss minimization,” in International Conference on Machine Learning

2018

Cited alongside, same era.

F. Sattler, S. Wiedemann, K.-R. Müller, and W. Samek, “Robust and communication-efficient federated learning from non-iid data,” IEEE transactions on neural networks and learning systems

2019

Cited alongside, same era.

M. Mohri, G. Sivek, and A. T. Suresh, “Agnostic federated learning,” in Proceedings of the 36th International Conference on Machine Learning

2019

Cited alongside, same era.

A. T. Suresh, B. McMahan, P. Kairouz, and Z. Sun, “Can you really backdoor federated learning?,” 2019

2019

Cited alongside, same era.

Y. Jiang, J. Konečnỳ, K. Rush, and S. Kannan, “Improving federated learning personalization via model agnostic meta learning,” arXiv preprint arXiv:1909.12488

Original

2019

Cited alongside, same era.

S. U. Stich, “Local SGD converges fast and communicates little,” in International Conference on Learning Representations

2019

Cited alongside, same era.

F. Sattler, K.-R. Müller, and W. Samek, “Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints,” IEEE Transactions on Neural Networks and Learning Systems

2020