Fetching the paper…
Reading the bibliography…
Large language models (LLMs) show amazing performance on many domain-specific tasks after fine-tuning with some appropriate data.
Large language models in medicine
Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023 · 1940
Earlier work this paper cites.
How transferable are features in deep neural networks?. In Proc. of Advances in neural information processing systems (NeurIPS’14)
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014 · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015 · 2015
Earlier work this paper cites.
Regulation (EU) 2016/679 of the European Parliament and of the Council
GDPR. 2016 · 2016
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data. In Proc. of Artificial intelligence and statistics (AISTAT’17) . 1273–1282
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017 · 2017
Earlier work this paper cites.
Decoupled Weight Decay Regularization. In Proc. of International Conference on Learning Representations (ICLR’18)
Ilya Loshchilov and Frank Hutter. 2018 · 2018
Earlier work this paper cites.
On the Convergence of FedAvg on Non-IID Data. In Proc. of International Conference on Learning Representations (ICLR’19)
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. 2019 · 2019
Earlier work this paper cites.
Federated Learning Based on Dynamic Regularization. In Proc. of International Conference on Learning Representations (ICLR’20)
Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. 2020 · 2020
Earlier work this paper cites.
Scaffold: Stochastic controlled averaging for federated learning. In Proc. of International conference on machine learning (ICML’20) . 5132–5143
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020 · 2020
Earlier work this paper cites.
Ensemble distillation for robust model fusion in federated learning. In Proc. of Advances in Neural Information Processing Systems (NeurIPS’20) . 2351–2363
Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. 2020 · 2020
Earlier work this paper cites.
Tackling the objective inconsistency problem in heterogeneous federated optimization. In Proc. of Advances in neural information processing systems (NeurIPS’20) . 7611–7623
Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. 2020 · 2020
Earlier work this paper cites.
Evaluating large language models trained on code
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al · 2021
Earlier work this paper cites.
Training verifiers to solve math word problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al · 2021
Earlier work this paper cites.
LoRA: Low-Rank Adaptation of Large Language Models. In Proc. of International Conference on Learning Representations (ICLR’21)
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al · 2021
Earlier work this paper cites.
The Power of Scale for Parameter-Efficient Prompt Tuning. In Proc. of the Conference on Empirical Methods in Natural Language Processing (EMNLP’21) . 3045–3059
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021 · 2021
Earlier work this paper cites.
Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Proc. of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL/IJNLP’21) . 4582–4597
Xiang Lisa Li and Percy Liang. 2021 · 2021
Earlier work this paper cites.
Finetuned Language Models are Zero-Shot Learners. In Proc. of International Conference on Learning Representations (ICLR’21)
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021 · 2021
Cited alongside, same era.
Parameterized knowledge transfer for personalized federated learning. In Proc. of Advances in Neural Information Processing Systems (NeurIPS’21) . 10092–10104
Jie Zhang, Song Guo, Xiaosong Ma, Haozhao Wang, Wenchao Xu, and Feijie Wu. 2021 · 2021
Cited alongside, same era.
Holistic evaluation of language models
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al · 2022
Cited alongside, same era.
Training language models to follow instructions with human feedback. In Proc. of Advances in Neural Information Processing Systems (NeurIPS’22) . 27730–27744
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Cited alongside, same era.
Joint prompt optimization of stacked llms using variational inference. In Proc. of Advances in Neural Information Processing Systems (NeurIPS’23)
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, and Nicolas Le Roux. 2023 · 2023
Later among the works it cites.
FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models
Jingwei Sun, Ziyue Xu, Hongxu Yin, Dong Yang, Daguang Xu, Yiran Chen, and Holger R Roth. 2023 · 2023
Later among the works it cites.
Stanford Alpaca: An Instruction-following LLaMA model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023 · 2023
Later among the works it cites.
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
FedKC: Federated knowledge composition for multilingual natural language understanding. In Proc. of the ACM Web Conference 2022 (WWW’22) . 1839–1850
Haoyu Wang, Handong Zhao, Yaqing Wang, Tong Yu, Jiuxiang Gu, and Jing Gao. 2022 · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models. In Proc. of Advances in Neural Information Processing Systems (NeurIPS’22) . 24824–24837
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al · 2022
Cited alongside, same era.
California Consumer Privacy Act (CCPA)
CCPA. 2023 · 2023
Cited alongside, same era.
Code Alpaca: An Instruction-following LLaMA model for code generation
Sahil Chaudhary. 2023 · 2023
Cited alongside, same era.
Free Dolly: Introducing the World’s First Truly Open Instruction-Tuned LLM
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023 · 2023
Cited alongside, same era.
Chatlaw: Open-source legal large language model with integrated external knowledge bases
Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Li Yuan. 2023 · 2023
Cited alongside, same era.
GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learning
Shiqi He, Qifan Yan, Feijie Wu, Lanjun Wang, Mathias Lécuyer, and Ivan Beschastnikh. 2023 · 2023
Cited alongside, same era.
Efficient federated prompt tuning for black-box large pre-trained models
Zihao Lin, Yan Sun, Yifan Shi, Xueqian Wang, Lifu Huang, Li Shen, and Dacheng Tao. 2023 · 2023
Cited alongside, same era.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023
Later among the works it cites.
Chatcad: Interactive computer-aided diagnosis on medical image using large language models
Sheng Wang, Zihao Zhao, Xi Ouyang, Qian Wang, and Dinggang Shen. 2023c · 2023
Later among the works it cites.
Anchor sampling for federated learning with partial client participation. In Proc. of International Conference on Machine Learning (ICML’23) . 37379–37416
Feijie Wu, Song Guo, Zhihao Qu, Shiqi He, Ziming Liu, and Jing Gao. 2023 · 2023
Later among the works it cites.
Offsite-tuning: Transfer learning without full model
Guangxuan Xiao, Ji Lin, and Song Han. 2023 · 2023
Later among the works it cites.
FederatedScope: A Flexible Federated Learning Platform for Heterogeneity. In Proc. of the VLDB Endowment (VLDB’23) . 1059–1072
Yuexiang Xie, Zhen Wang, Dawei Gao, Daoyuan Chen, Liuyi Yao, Weirui Kuang, Yaliang Li, Bolin Ding, and Jingren Zhou. 2023 · 2023
Later among the works it cites.
Fedlora: Model-heterogeneous personalized federated learning with lora tuning
Liping Yi, Han Yu, Gang Wang, and Xiaoguang Liu. 2023 · 2023
Later among the works it cites.
FedPETuning: When federated learning meets the parameter-efficient tuning methods of pre-trained language models. In Proc. of Annual Meeting of the Association of Computational Linguistics (ACL’23) . 9963–9977
Zhuo Zhang, Yuanhang Yang, Yong Dai, Qifan Wang, Yue Yu, Lizhen Qu, and Zenglin Xu. 2023 · 2023
Later among the works it cites.
Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In Proc. of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’23) . 5673–5684
Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Lei Shen, Zihan Wang, Andi Wang, Yang Li, et al · 2023
Later among the works it cites.
FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning. In Proc. of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’24)
Weirui Kuang, Bingchen Qian, Zitao Li, Daoyuan Chen, Dawei Gao, Xuchen Pan, Yuexiang Xie, Yaliang Li, Bolin Ding, and Jingren Zhou. 2024 · 2024
Closest in time.
Large language models as tax attorneys: a case study in legal capabilities emergence
John J Nay, David Karamardian, Sarah B Lawsky, Wenting Tao, Meghana Bhat, Raghav Jain, Aaron Travis Lee, Jonathan H Choi, and Jungo Kasai. 2024 · 2024
Closest in time.
Improving LoRA in Privacy-preserving Federated Learning. In Proc. of The International Conference on Learning Representations (ICLR’24)
Youbang Sun, Zitao Li, Yaliang Li, and Bolin Ding. 2024 · 2024
Closest in time.
Towards building the federatedGPT: Federated instruction tuning. In Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’24) . 6915–6919
Jianyi Zhang, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Tong Yu, Guoyin Wang, and Yiran Chen. 2024 · 2024
Closest in time.