Fetching the paper…
Reading the bibliography…
While "instruction-tuned" generative large language models (LLMs) have demonstrated an impressive ability to generalize to new tasks, the training phases heavily rely on large amounts of diverse and high-quality instruction data (such as ChatGPT and GPT-4).
Accelerating drug discovery
Sandra Kraljevic, Peter J Stambrook, and Kreśimir Pavelic · 2004
Earlier work this paper cites.
Bayesian learning via stochastic gradient Langevin dynamics
M. Welling and Y. W. Teh · 2011
Earlier work this paper cites.
Deep learning with differential privacy
Martín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.
Stein variational gradient descent: A general purpose Bayesian inference algorithm
Q. Liu and D. Wang · 2016
Earlier work this paper cites.
Deep reinforcement learning from human preferences
Paul Francis Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei · 2017
Earlier work this paper cites.
Mednn: A distributed mobile system with enhanced partition and deployment for large-scale dnns
Jiachen Mao, Zhongda Yang, Wei Wen, Chunpeng Wu, Linghao Song, Kent W. Nixon, Xiang Chen, Hai Li, and Yiran Chen · 2017
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas · 2017
Earlier work this paper cites.
Communication-efficient Learning of Deep Networks from Decentralized Data
H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al · 2017
Earlier work this paper cites.
Leaf: A benchmark for federated settings
Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečnỳ, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Earlier work this paper cites.
Federated learning for mobile keyboard prediction
Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al · 2018
Earlier work this paper cites.
Federated optimization for heterogeneous networks
Anit Kumar Sahu, Tian Li, Maziar Sanjabi, Manzil Zaheer, Ameet Talwalkar, and Virginia Smith · 2018
Earlier work this paper cites.
Glue: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman · 2018
Earlier work this paper cites.
Applied federated learning: Improving google keyboard query suggestions
Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, and Françoise Beaufays · 2018
Earlier work this paper cites.
Cognitive graph for multi-hop reading comprehension at scale
Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang · 2019
Earlier work this paper cites.
Jack Goetz, Kshitiz Malik, Duc Bui, Seungwhan Moon, Honglei Liu, and Anuj Kumar · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
Cyclical stochastic gradient mcmc for bayesian deep learning
Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, and Andrew Gordon Wilson · 2019
Earlier work this paper cites.
Self-adversarially learned bayesian sampling
Yang Zhao, Jianyi Zhang, and Changyou Chen · 2019
Earlier work this paper cites.
Deep leakage from gradients
Ligeng Zhu, Zhijian Liu, and Song Han · 2019
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Fedmax: mitigating activation divergence for accurate and communication-efficient federated learning
Wei Chen, Kartikeya Bhardwaj, and Radu Marculescu · 2020
Earlier work this paper cites.
Yae Jee Cho, Jianyu Wang, and Gauri Joshi · 2020
Earlier work this paper cites.
Heterofl: Computation and communication efficient federated learning for heterogeneous clients
Enmao Diao, Jie Ding, and Vahid Tarokh · 2020
Earlier work this paper cites.
Personalized federated learning: A meta-learning approach
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar · 2020
Earlier work this paper cites.
Fedner: Privacy-preserving medical named entity recognition with federated learning
Suyu Ge, Fangzhao Wu, Chuhan Wu, Tao Qi, Yongfeng Huang, and Xing Xie · 2020
Earlier work this paper cites.
Fedml: A research library and benchmark for federated machine learning
Chaoyang He, Songze Li, Jinhyun So, Xiao Zeng, Mi Zhang, Hongyi Wang, Xiaoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, et al · 2020
Earlier work this paper cites.
Ang Li, Jingwei Sun, Binghui Wang, Lin Duan, Sicheng Li, Yiran Chen, and Hai Li · 2020
Earlier work this paper cites.
Counterfactual Theories of Causation
Peter Menzies and Helen Beebee · 2020
Earlier work this paper cites.
Adaptive Federated Optimization
Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H. Brendan McMahan · 2020
Earlier work this paper cites.
Amirhossein Reisizadeh, Isidoros Tziotis, Hamed Hassani, Aryan Mokhtari, and Ramtin Pedarsani · 2020
Earlier work this paper cites.
Learning to summarize with human feedback
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano · 2020
Earlier work this paper cites.
Feded: Federated learning via ensemble distillation for medical relation extraction
Dianbo Sui, Yubo Chen, Jun Zhao, Yantao Jia, Yuantao Xie, and Weijian Sun · 2020
Earlier work this paper cites.
Stochastic particle-optimization sampling and the non-asymptotic convergence theory
Jianyi Zhang, Ruiyi Zhang, Lawrence Carin, and Changyou Chen · 2020
Cited alongside, same era.
Variance reduction in stochastic particle-optimization sampling
Jianyi Zhang, Yang Zhao, and Changyou Chen · 2020
Cited alongside, same era.
Omid Aramoon, Pin-Yu Chen, Gang Qu, and Yuan Tian · 2021
Cited alongside, same era.
Privacy enabled financial text classification using differential privacy and federated learning
Priyam Basu, Tiasa Singha Roy, Rakshit Naidu, and Zümrüt Müftüoglu · 2021
Cited alongside, same era.
On convergence of federated averaging langevin dynamics
Wei Deng, Qian Zhang, Yi-An Ma, Zhao Song, and Guang Lin · 2021
A comprehensive survey on federated learning: Concept and applications
Dhurgham Hassan Mahlool and Mohammed Hamzah Abed · 2022
Later among the works it cites.
Peft: State-of-the-art parameter-efficient fine-tuning methods
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, and Sayak Paul Younes Belkada · 2022
Later among the works it cites.
Cross-task generalization via natural language crowdsourcing instructions
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi · 2022
Later among the works it cites.
Introducing chatgpt
OpenAI · 2022
Later among the works it cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Later among the works it cites.
Federated learning from pre-trained models: A contrastive learning approach
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Towards fair federated learning with zero-shot data augmentation
Weituo Hao, Mostafa El-Khamy, Jungwon Lee, Jianyi Zhang, Kevin J Liang, Changyou Chen, and Lawrence Carin Duke · 2021
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Cited alongside, same era.
An investigation towards differentially private sequence tagging in a federated framework
Abhik Jana and Chris Biemann · 2021
Cited alongside, same era.
Advances and open problems in federated learning
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al · 2021
Cited alongside, same era.
Oort: Efficient federated learning via guided participant selection
Fan Lai, Xiangfeng Zhu, Harsha V. Madhyastha, and Mosharaf Chowdhury · 2021
Cited alongside, same era.
The power of scale for parameter-efficient prompt tuning
Brian Lester, Rami Al-Rfou, and Noah Constant · 2021
Cited alongside, same era.
Hermes: An efficient federated learning framework for heterogeneous mobile clients
Ang Li, Jingwei Sun, Pengcheng Li, Yu Pu, Hai Li, and Yiran Chen · 2021
Cited alongside, same era.
Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, and Jing Jiang · 2022
Later among the works it cites.
Fade: Enabling large-scale federated adversarial training on resource-constrained edge devices
Minxue Tang, Jianyi Zhang, Mingyuan Ma, Louis DiValentin, Aolin Ding, Amin Hassanzadeh, Hai Li, and Yiran Chen · 2022
Later among the works it cites.
Fedbert: When federated learning meets pre-training
Yuanyishu Tian, Yao Wan, Lingjuan Lyu, Dezhong Yao, Hai Jin, and Lichao Sun · 2022
Later among the works it cites.
When do curricula work in federated learning?
Saeed Vahidian, Sreevatsank Kadaveru, Woonjoon Baek, Weijia Wang, Vyacheslav Kungurtsev, Chen Chen, Mubarak Shah, and Bill Lin · 2022
Later among the works it cites.
Saeed Vahidian, Mahdi Morafah, Weijia Wang, Vyacheslav Kungurtsev, Chen Chen, Mubarak Shah, and Bill Lin · 2022
Later among the works it cites.
Fedkc: Federated knowledge composition for multilingual natural language understanding
Haoyu Wang, Handong Zhao, Yaqing Wang, Tong Yu, Jiuxiang Gu, and Jing Gao · 2022
Later among the works it cites.
Self-instruct: Aligning language model with self generated instructions
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi · 2022
Later among the works it cites.
Pretrained models for multilingual federated learning
Orion Weller, Marc Marone, Vladimir Braverman, Dawn Lawrie, and Benjamin Van Durme · 2022
Later among the works it cites.
Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models, 2022
Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg · 2022
Later among the works it cites.
Next generation federated learning for edge devices: An overview
Jianyi Zhang, Zhixu Du, Jingwei Sun, Ang Li, Minxue Tang, Yuhao Wu, Zhihui Gao, Martin Kuo, Hai Helen Li, and Yiran Chen · 2022
Later among the works it cites.
Next generation federated learning for edge devices: An overview
Jianyi Zhang, Zhixu Du, Jingwei Sun, Ang Li, Minxue Tang, Yuhao Wu, Zhihui Gao, Martin Kuo, Hai-Helen Li, and Yiran Chen · 2022
Later among the works it cites.
Pythia: A suite for analyzing large language models across training and scaling, 2023
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal · 2023
Closest in time.
Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing · 2023
Closest in time.
Meta policy learning for cold-start conversational recommendation
Zhendong Chu, Hongning Wang, Yun Xiao, Bo Long, and Lingfei Wu · 2023
Closest in time.
Improving a named entity recognizer trained on noisy data with a few clean instances
Zhendong Chu, Ruiyi Zhang, Tong Yu, Rajiv Jain, Vlad I Morariu, Jiuxiang Gu, and Ani Nenkova · 2023
Closest in time.
Koala: A dialogue model for academic research
Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song · 2023
Closest in time.
Samsung bans staff’s ai use after spotting chatgpt data leak, May 2023
Mark Gurman · 2023
Closest in time.
Visual instruction tuning, 2023
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee · 2023
Closest in time.
Flis: Clustered federated learning via inference similarity for non-iid data distribution
Mahdi Morafah, Saeed Vahidian, Weijia Wang, and Bill Lin · 2023
Closest in time.
OpenAI · 2023
Closest in time.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao · 2023
Closest in time.
Stanford alpaca: An instruction-following llama model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto · 2023
Closest in time.
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023
Closest in time.
Federated few-shot learning
Song Wang, Xingbo Fu, Kaize Ding, Chen Chen, Huiyuan Chen, and Jundong Li · 2023
Closest in time.
Baize: An open-source chat model with parameter-efficient tuning on self-chat data
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley · 2023
Closest in time.
Shepherd: Large language models with parameter-efficient federated finetuning in the presence of heterogeneous instructions
Jianyi Zhang, Martin Kuo, Ruiyi Zhang, Guoyin Wang, Saeed Vahidian, and Yiran Chen · 2023
Closest in time.
Fed-cbs: A heterogeneity-aware client sampling mechanism for federated learning via class-imbalance reduction
Jianyi Zhang, Ang Li, Minxue Tang, Jingwei Sun, Xiang Chen, Fan Zhang, Changyou Chen, Yiran Chen, and Hai Li · 2023
Closest in time.
Llama-adapter: Efficient fine-tuning of language models with zero-init attention
Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Qiao Yu · 2023
Closest in time.