DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism, March 2020

Original

Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro · 1909

Earlier work this paper cites.

ViGGO: A Video Game Corpus for Data-To-Text Generation in Open-Domain Conversation, October 2019

Original

Juraj Juraska, Kevin K. Bowden, and Marilyn Walker · 1910

Earlier work this paper cites.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, September 2023

Original

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu · 1910

Earlier work this paper cites.

Optimal Brain Damage

Yann LeCun, John Denker, and Sara Solla · 1989

Earlier work this paper cites.

Optimal Brain Surgeon and general network pruning

B. Hassibi, D.G. Stork, and G.J. Wolff · 1993

Earlier work this paper cites.

TurboTransformers: An Efficient GPU Serving System For Transformer Models, February 2021

Original

Jiarui Fang, Yang Yu, Chengduo Zhao, and Jie Zhou · 2010

Earlier work this paper cites.

CUDA Dynamic Parallelism API and Principles, May 2014

Adinets Andy · 2014

Earlier work this paper cites.

Sequence to sequence learning with neural networks

Ilya Sutskever, Oriol Vinyals, and Quoc V Le · 2014

Earlier work this paper cites.

Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding

Original

Song Han, Huizi Mao, and William J Dally · 2015

Earlier work this paper cites.

Google’s neural machine translation system: Bridging the gap between human and machine translation

Original

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al · 2016

Earlier work this paper cites.

Clipper: A {Low-Latency} Online Prediction Serving System

Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J. Franklin, Joseph E. Gonzalez, and Ion Stoica · 2017

Earlier work this paper cites.

Serving DNNs like Clockwork: Performance Predictability from the Bottom Up

Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, and Jonathan Mace · 2020

Earlier work this paper cites.

Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider

Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini · 2020

Earlier work this paper cites.

Transformers: State-of-the-Art Natural Language Processing

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush · 2020

Earlier work this paper cites.

Evaluating Large Language Models Trained on Code, July 2021

Original

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba · 2021

Earlier work this paper cites.

LoRA: Low-Rank Adaptation of Large Language Models, October 2021

Original

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021

Earlier work this paper cites.

Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks

Itay Hubara, Brian Chmiel, Moshe Island, Ron Banner, Joseph Naor, and Daniel Soudry · 2021

Earlier work this paper cites.

NVCOMP, March 2021

NVIDIA · 2021

Earlier work this paper cites.

{INFaaS}: Automated Model-less Inference Serving

Francisco Romero, Qian Li, Neeraja J. Yadwadkar, and Christos Kozyrakis · 2021

Earlier work this paper cites.

Faster and Cheaper Serverless Computing on Harvested Resources

Yanqi Zhang, Inigo Goiri, Gohar Irfan Chaudhry, Rodrigo Fonseca, Sameh Elnikety, Christina Delimitrou, and Ricardo Bianchini · 2021

Earlier work this paper cites.

Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch, April 2021

Original

Aojun Zhou, Yukun Ma, Junnan Zhu, Jianbo Liu, Zhijie Zhang, Kun Yuan, Wenxiu Sun, and Hongsheng Li · 2021

Earlier work this paper cites.

Cuda toolkit 12.0 released for general availability

Rob Armstrong, Rob Nertney, Arthy Sundaram, Matthew Nicely, Robert Jensen, and Fred Oh · 2022

Earlier work this paper cites.

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness, June 2022

Original

Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré · 2022

Earlier work this paper cites.

xformers: A modular and hackable transformer modelling library

Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, Daniel Haziza, Luca Wehrstedt, Jeremy Reizenstein, and Grigory Sizov · 2022

Earlier work this paper cites.

Cross-Task Generalization via Natural Language Crowdsourcing Instructions, March 2022

Original

Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi · 2022

Earlier work this paper cites.

Training language models to follow instructions with human feedback, March 2022

Original

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe · 2022

Earlier work this paper cites.

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers, June 2022

Original

Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He · 2022

Earlier work this paper cites.

Orca: A Distributed Serving System for {Transformer-Based} Generative Models

Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun · 2022

Earlier work this paper cites.

{PetS}: A Unified Framework for {Parameter-Efficient} Transformers Serving

Zhe Zhou, Xuechao Wei, Jiejing Zhang, and Guangyu Sun · 2022

Earlier work this paper cites.

https://github.com/ray-project/llm-numbers , 2023

Numbers every llm developer should know · 2023

Earlier work this paper cites.

https://www.nvidia.com/en-us/data-center/nvlink/ , 2023

Nvlink and nvswitch · 2023

Cited alongside, same era.

Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2, November 2023

Anyscale · 2023

Cited alongside, same era.

sql-create-context dataset, 2023

b mc2 · 2023

Cited alongside, same era.

Structured Sparsity in the NVIDIA Ampere Architecture and Applications in Search Engines, July 2023

Hongxiao Bai and Yun Li · 2023

Cited alongside, same era.

INT2.1: Towards Fine-Tunable Quantized Large Language Models with Error Correction through Low-Rank Adaptation, June 2023

Original

Yuji Chai, John Gkountouras, Glenn G. Ko, David Brooks, and Gu-Yeon Wei · 2023

Cited alongside, same era.

Punica: Multi-Tenant LoRA Serving, October 2023

https://github.com/meta-llama/llama3 , 2024

Meta llama 3 · 2024

Closest in time.

Lora learns less and forgets less, 2024

Dan Biderman, Jose Gonzalez Ortiz, Jacob Portes, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John P. Cunningham · 2024

Closest in time.

Sparse-marlin: Boosting 4-bit inference kernels with 2:4 sparsity

Roberto L. Castro and Dan Alistarh · 2024

Closest in time.

Punica: Multi-tenant lora serving

Lequn Chen, Zihao Ye, Yongji Wu, Danyang Zhuo, Luis Ceze, and Arvind Krishnamurthy · 2024

Closest in time.

MuxServe: Flexible Multiplexing for Efficient Multiple LLM Serving, April 2024

Original

Jiangfei Duan, Runyu Lu, Haojie Duanmu, Xiuhong Li, Xingcheng Zhang, Dahua Lin, Ion Stoica, and Hao Zhang · 2024

Closest in time.

Marlin: a fast 4-bit inference kernel for medium batchsizes