eFedLLM: Efficient LLM Inference Based on Federated Learning

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901

Earlier work this paper cites.

A zero knowledge sumcheck and its applications

Original

Alessandro Chiesa, Michael A Forbes, and Nicholas Spooner. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Bert: Pre-training of deep bidirectional transformers for language understanding

Original

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018

Earlier work this paper cites.

Understanding distributed poisoning attack in federated learning. In 2019 IEEE 25th international conference on parallel and distributed systems (ICPADS) . IEEE, IEEE, Tianjin, China, 233–239

Di Cao, Shan Chang, Zhijian Lin, Guohua Liu, and Donghong Sun. 2019 · 2019

Earlier work this paper cites.

Parameter-efficient transfer learning for NLP. In International conference on machine learning . PMLR, ICML, Long Beach, CA, USA, 2790–2799

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019 · 2019

Earlier work this paper cites.

Advances in neural information processing systems

He Lyu, Ningyu Sha, Shuyang Qin, Ming Yan, Yuying Xie, and Rongrong Wang. 2019 · 2019

Earlier work this paper cites.

A hybrid approach to privacy-preserving federated learning. In Proceedings of the 12th ACM workshop on artificial intelligence and security . ACM, New York, NY, United States, 1–11

Stacey Truex, Nathalie Baracaldo, Ali Anwar, Thomas Steinke, Heiko Ludwig, Rui Zhang, and Yi Zhou. 2019 · 2019

Earlier work this paper cites.

Adaptive federated learning in resource constrained edge computing systems

Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K Leung, Christian Makaya, Ting He, and Kevin Chan. 2019 · 2019

Earlier work this paper cites.

Local model poisoning attacks to { \{ Byzantine-Robust } \} federated learning. In 29th USENIX security symposium (USENIX Security 20) . 1605–1622

Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. 2020 · 2020

Earlier work this paper cites.

Scaling laws for neural language models

Original

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020 · 2020

Cited alongside, same era.

Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . IEEE, Seattle, Washington, USA, 8110–8119

Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2020 · 2020

Cited alongside, same era.

A review of applications in federated learning

Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. 2020 · 2020

Cited alongside, same era.

Communication-efficient federated learning

Mingzhe Chen, Nir Shlezinger, H. Vincent Poor, Yonina C. Eldar, and Shuguang Cui. 2021 · 2021

Cited alongside, same era.

Lora: Low-rank adaptation of large language models

Original

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

An Investigation of Smart Contract for Collaborative Machine Learning Model Training

Original

Shengwen Ding and Chenhui Hu. 2022a · 2022

Later among the works it cites.

Refl: Resource-efficient federated learning. In Proceedings of the Eighteenth European Conference on Computer Systems . ACM, Rome, Italy, 215–232

Ahmed M Abdelmoniem, Atal Narayan Sahu, Marco Canini, and Suhaib A Fahmy. 2023 · 2023

Later among the works it cites.

Fast algorithms for singular value decomposition and the inverse of nearly low-rank matrices

Chen Xu, Weiwei Xu, and Kaili Jing. 2023 · 2023

Later among the works it cites.

Phi-3 technical report: A highly capable language model locally on your phone

Original

Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al · 2024

Closest in time.

Medusa: Simple llm inference acceleration framework with multiple decoding heads