Fetching the paper…
Reading the bibliography…
Recently, foundation models, particularly large language models (LLMs), have demonstrated an impressive ability to adapt to various tasks by fine-tuning diverse instruction data.
Mixture of experts: a literature survey
Saeed Masoudnia and Reza Ebrahimpour · 2014
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas · 2017
Earlier work this paper cites.
Parameter-efficient transfer learning for nlp
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly · 2019
Earlier work this paper cites.
Out of distribution generalization in machine learning
Martin Arjovsky · 2020
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar · 2020
Earlier work this paper cites.
Federated optimization in heterogeneous networks
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith · 2020
Earlier work this paper cites.
Exploiting shared representations for personalized federated learning
Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai · 2021
Earlier work this paper cites.
Towards a unified view of parameter-efficient transfer learning
Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig · 2021
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Earlier work this paper cites.
The power of scale for parameter-efficient prompt tuning
Brian Lester, Rami Al-Rfou, and Noah Constant · 2021
Earlier work this paper cites.
Ditto: Fair and robust federated learning through personalization
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith · 2021
Earlier work this paper cites.
Prefix-tuning: Optimizing continuous prompts for generation
Xiang Lisa Li and Percy Liang · 2021
Earlier work this paper cites.
Fedbn: Federated learning on non-iid features via local batch normalization
Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou · 2021
Earlier work this paper cites.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Earlier work this paper cites.
Finetuned language models are zero-shot learners
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le · 2021
Cited alongside, same era.
Federated continual learning with weighted inter-client transfer
Jaehong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, and Sung Ju Hwang · 2021
Cited alongside, same era.
Federated class-incremental learning
Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu · 2022
Cited alongside, same era.
Krona: Parameter efficient tuning with kronecker adapter
Ali Edalati, Marzieh Tahaei, Ivan Kobyzev, Vahid Partovi Nia, James J Clark, and Mehdi Rezagholizadeh · 2022
Cited alongside, same era.
Towards personalized federated learning
Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang · 2022
Cited alongside, same era.
Fedlora: Model-heterogeneous personalized federated learning with lora tuning
Liping Yi, Han Yu, Gang Wang, and Xiaoguang Liu · 2023
Later among the works it cites.
Federated foundation models: Privacy-preserving and collaborative learning for large models
Sixing Yu, J Pablo Muñoz, and Ali Jannesari · 2023
Later among the works it cites.
Towards building the federated gpt: Federated instruction tuning
Jianyi Zhang, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Guoyin Wang, and Yiran Chen · 2023
Later among the works it cites.
Fedpetuning: When federated learning meets the parameter-efficient tuning methods of pre-trained language models
Zhuo Zhang, Yuanhang Yang, Yong Dai, Qifan Wang, Yue Yu, Lizhen Qu, and Zenglin Xu · 2023
Later among the works it cites.
When foundation model meets federated learning: Motivations, challenges, and future directions
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Sara Babakniya, Ahmed Roushdy Elkordy, Yahya H Ezzeldin, Qingfeng Liu, Kee-Bong Song, Mostafa El-Khamy, and Salman Avestimehr · 2023
Cited alongside, same era.
Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models
Zhiqiang Hu, Yihuai Lan, Lei Wang, Wanyu Xu, Ee-Peng Lim, Roy Ka-Wei Lee, Lidong Bing, and Soujanya Poria · 2023
Cited alongside, same era.
Low-parameter federated learning with large language models
Jingang Jiang, Xiangyang Liu, and Chenyou Fan · 2023
Cited alongside, same era.
Test-time robust personalization for federated learning
Liangze Jiang and Tao Lin · 2023
Cited alongside, same era.
Weirui Kuang, Bingchen Qian, Zitao Li, Daoyuan Chen, Dawei Gao, Xuchen Pan, Yuexiang Xie, Yaliang Li, Bolin Ding, and Jingren Zhou · 2023
Cited alongside, same era.
Fedbpt: Efficient federated black-box prompt tuning for large language models
Jingwei Sun, Ziyue Xu, Hongxu Yin, Dong Yang, Daguang Xu, Yiran Chen, and Holger R Roth · 2023
Cited alongside, same era.
How far can camels go? exploring the state of instruction tuning on open resources
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al · 2023
Cited alongside, same era.
Weiming Zhuang, Chen Chen, and Lingjuan Lyu · 2023
Later among the works it cites.
Adaptive test-time personalization for federated learning
Wenxuan Bao, Tianxin Wei, Haohan Wang, and Jingrui He · 2024
Closest in time.
Towards federated foundation models: Scalable dataset pipelines for group-structured learning
Zachary Charles, Nicole Mitchell, Krishna Pillutla, Michael Reneer, and Zachary Garrett · 2024
Closest in time.
Feddat: An approach for foundation model finetuning in multi-modal heterogeneous federated learning
Haokun Chen, Yao Zhang, Denis Krompass, Jindong Gu, and Volker Tresp · 2024
Closest in time.
Advances and open challenges in federated learning with foundation models
Chao Ren, Han Yu, Hongyi Peng, Xiaoli Tang, Anran Li, Yulan Gao, Alysa Ziying Tan, Bo Zhao, Xiaoxiao Li, Zengxiang Li, et al · 2024
Closest in time.
Is heterogeneity notorious? taming heterogeneity to handle test-time shift in federated learning
Yue Tan, Chen Chen, Weiming Zhuang, Xin Dong, Lingjuan Lyu, and Guodong Long · 2024
Closest in time.
Fwdllm: Efficient fedllm using forward gradient
M Xu, D Cai, Y Wu, X Li, and S Wang · 2024
Closest in time.
Eva: Zero-shot accurate attributes and multi-object video editing
Xiangpeng Yang, Linchao Zhu, Hehe Fan, and Yi Yang · 2024
Closest in time.
Dgl: Dynamic global-local prompt tuning for text-video retrieval
Xiangpeng Yang, Linchao Zhu, Xiaohan Wang, and Yi Yang · 2024
Closest in time.
Jun Yu, Yutong Dai, Xiaokang Liu, Jin Huang, Yishan Shen, Ke Zhang, Rong Zhou, Eashan Adhikarla, Wenxuan Ye, Yixin Liu, et al · 2024
Closest in time.