Confidant: Customizing Transformer-based LLMs via Collaborative Edge Training

Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition

Erik F Sang and Fien De Meulder. 2003 · 2003

Earlier work this paper cites.

Training deep nets with sublinear memory cost

Original

Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016 · 2016

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Bert: Pre-training of deep bidirectional transformers for language understanding

Original

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018

Earlier work this paper cites.

PipeDream: Generalized pipeline parallelism for DNN training. In SOSP

Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R Devanur, Gregory R Ganger, Phillip B Gibbons, and Matei Zaharia. 2019 · 2019

Cited alongside, same era.

MNN: A Universal and Efficient Inference Engine. In MLSys

Xiaotang Jiang, Huan Wang, Yiliu Chen, Ziqi Wu, Lichuan Wang, Bin Zou, Yafeng Yang, Zongyang Cui, Yu Cai, Tianhang Yu, Chengfei Lv, and Zhihua Wu. 2020 · 2020

Cited alongside, same era.

Codl: efficient cpu-gpu co-execution for deep learning inference on mobile devices. In Mobisys

Fucheng Jia, Deyu Zhang, Ting Cao, Shiqi Jiang, Yunxin Liu, Ju Ren, and Yaoxue Zhang. 2022 · 2022

Cited alongside, same era.

FTPipeHD: A Fault-Tolerant Pipeline-Parallel Distributed Training Approach for Heterogeneous Edge Devices

Yuhao Chen, Qianqian Yang, Shibo He, Zhiguo Shi, Jiming Chen, and Mohsen Guizani. 2023 · 2023

Cited alongside, same era.

Melon: Breaking the memory wall for resource-efficient on-device machine learning. In Mobisys

Qipeng Wang, Mengwei Xu, Chao Jin, Xinran Dong, Jinliang Yuan, Xin Jin, Gang Huang, Yunxin Liu, and Xuanzhe Liu. 2022b

Cited in the paper.

Lightseq2: Accelerated training for transformer-based models on gpus. In SC

Xiaohui Wang, Yang Wei, Ying Xiong, Guyue Huang, Xian Qian, Yufei Ding, Mingxuan Wang, and Lei Li. 2022a

Cited in the paper.

GPT-4 Technical Report

Original

OpenAI. 2023 · 2023

Closest in time.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023

Closest in time.

Java Native Interface — Wikipedia, The Free Encyclopedia

Wikipedia contributors. 2023 · 2023

Closest in time.

EdgeMoE: Fast On-Device Inference of MoE-based Large Language Models

Original

Rongjie Yi, Liwei Guo, Shiyun Wei, Ao Zhou, Shangguang Wang, and Mengwei Xu. 2023 · 2023

Closest in time.