Fetching the paper…
Reading the bibliography…
Foundation Models (FMs), such as LLaMA, BERT, GPT, ViT, and CLIP, have demonstrated remarkable success in a wide range of applications, driven by their ability to leverage vast amounts of data for pre-training.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901
Earlier work this paper cites.
Adaptive personalized federated learning
Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. 2020 · 2003
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020 · 2010
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017 · 2017
Earlier work this paper cites.
Federated learning with non-iid data
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018 · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019
Earlier work this paper cites.
Cmfl: Mitigating communication overhead for federated learning
Luping WANG, Wei WANG, and Bo LI. 2019 · 2019
Earlier work this paper cites.
Scaffold: Stochastic controlled averaging for federated learning
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020 · 2020
Earlier work this paper cites.
Ensemble distillation for robust model fusion in federated learning
Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. 2020 · 2020
Earlier work this paper cites.
Accelerating federated learning via momentum gradient descent
Wei Liu, Li Chen, Yunfei Chen, and Wenyi Zhang. 2020 · 2020
Earlier work this paper cites.
Towards asynchronous federated learning for heterogeneous edge-powered internet of things. digit commun netw 7 (3): 317–326
Z Chen, W Liao, K Hua, C Lu, and W Yu. 2021 · 2021
Earlier work this paper cites.
Towards scalable simulation of federated learning
Tomasz Kołodziej and Paweł Rościszewski. 2021 · 2021
Earlier work this paper cites.
The power of scale for parameter-efficient prompt tuning
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021 · 2021
Earlier work this paper cites.
What makes good in-context examples for gpt- 3 3 ?
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021 · 2021
Earlier work this paper cites.
Toward node liability in federated learning: Computational cost and network overhead
Francesco Malandrino and Carla Fabiana Chiasserini. 2021 · 2021
Earlier work this paper cites.
Metaicl: Learning to learn in context
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2021 · 2021
Cited alongside, same era.
Differential privacy meets federated learning under communication constraints
Nima Mohammadi, Jianan Bai, Qiang Fan, Yifei Song, Yang Yi, and Lingjia Liu. 2021 · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021 · 2021
Cited alongside, same era.
Learning to retrieve prompts for in-context learning
Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2021 · 2021
Cited alongside, same era.
Multitask prompted training enables zero-shot task generalization
Spatl: salient parameter aggregation and transfer learning for heterogeneous federated learning
Sixing Yu, Phuong Nguyen, Waqwoya Abebe, Wei Qian, Ali Anwar, and Ali Jannesari. 2022a · 2022
Later among the works it cites.
Local training and scalability of federated learning systems
Syed Zawad, Feng Yan, and Ali Anwar. 2022 · 2022
Later among the works it cites.
Federated multi-task learning with non-stationary heterogeneous data
Hongwei Zhang, Meixia Tao, Yuanming Shi, and Xiaoyan Bi. 2022a · 2022
Later among the works it cites.
Security and privacy threats to federated learning: Issues, methods, and challenges
Junpeng Zhang, Hui Zhu, Fengwei Wang, Jiaqi Zhao, Qi Xu, Hui Li, et al. 2022b · 2022
Later among the works it cites.
Learning to prompt for vision-language models
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022 · 2022
Later among the works it cites.
Optimising communication overhead in federated learning using nsga-ii
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2021 · 2021
Cited alongside, same era.
Finetuned language models are zero-shot learners
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021 · 2021
Cited alongside, same era.
Adaptive dynamic pruning for non-iid federated learning
Sixing Yu, Phuong Nguyen, Ali Anwar, and Ali Jannesari. 2021 · 2021
Cited alongside, same era.
Ptr: Prompt tuning with rules for text classification
Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. 2022 · 2022
Cited alongside, same era.
LoRA: Low-rank adaptation of large language models
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022 · 2022
Cited alongside, same era.
Model pruning enables efficient federated learning on edge devices
Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K Leung, and Leandros Tassiulas. 2022 · 2022
Cited alongside, same era.
Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives
Pengrui Liu, Xiangrui Xu, and Wei Wang. 2022 · 2022
Cited alongside, same era.
Privacy and robustness in federated learning: Attacks and defenses
Lingjuan Lyu, Han Yu, Xingjun Ma, Chen Chen, Lichao Sun, Jun Zhao, Qiang Yang, and Philip S. Yu. 2022 · 2022
Cited alongside, same era.
José Ángel Morell, Zakaria Abdelmoiz Dahi, Francisco Chicano, Gabriel Luque, and Enrique Alba. 2022 · 2022
Later among the works it cites.
Communication and computation efficiency in federated learning: A survey
Omair Rashed Abdulwareth Almanifi, Chee-Onn Chow, Mau-Luen Tham, Joon Huang Chuah, and Jeevan Kanesan. 2023 · 2023
Closest in time.
Study of the performance and scalability of federated learning for medical imaging with intermittent clients
Judith Sáinz-Pardo Díaz and Álvaro López García. 2023 · 2023
Closest in time.
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. 2023 · 2023
Closest in time.
Textbooks are all you need ii: phi-1.5 technical report
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023 · 2023
Closest in time.
Measuring overhead costs of federated learning systems by eavesdropping
Rainer Meindl and Bernhard A Moser. 2023 · 2023
Closest in time.
Foundation model is efficient multimodal multitask model selector
Fanqing Meng, Wenqi Shao, Zhanglin Peng, Chonghe Jiang, Kaipeng Zhang, Yu Qiao, and Ping Luo. 2023 · 2023
Closest in time.
Enhancing heterogeneous federated learning with knowledge extraction and multi-model fusion
Duy Phuong Nguyen, Sixing Yu, J. Pablo Muñoz, and Ali Jannesari. 2023 · 2023
Closest in time.
OpenAI. 2023 · 2023
Closest in time.
Lonas: Elastic low-rank adapters for efficient large language models
J. Pablo Muñoz, Jinjie Yuan, Yi Zheng, and Nilesh Jain. 2024b · 2024
Closest in time.