BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems

Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining . 785–794

Tianqi Chen and Carlos Guestrin. 2016 · 2016

Earlier work this paper cites.

Fast transformer decoding: One write-head is all you need

Original

Noam Shazeer. 2019 · 2019

Earlier work this paper cites.

Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 3505–3506

Jeff Rasley and Rajbhandari et.al. 2020 · 2020

Earlier work this paper cites.

Faster and Cheaper Serverless Computing on Harvested Resources. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles

Yanqi Zhang, Íñigo Goiri, Gohar Irfan Chaudhry, Rodrigo Fonseca, Sameh Elnikety, Christina Delimitrou, and Ricardo Bianchini. 2021 · 2021

Earlier work this paper cites.

Llm. int8 (): 8-bit matrix multiplication for transformers at scale

Original

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022 · 2022

Earlier work this paper cites.

Introducing ChatGPT

OpenAI. 2022 · 2022

Earlier work this paper cites.

Orca: A Distributed Serving System for Transformer-Based Generative Models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) . USENIX Association, Carlsbad, CA, 521–538

Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022 · 2022

Earlier work this paper cites.

Lightllm: A python-based large language model inference and serving framework

2023 · 2023

Earlier work this paper cites.

Failure Analysis of Big Cloud Service Providers Prior to and During Covid-19 Period

Original

Muhammad Ahsan, Sacheendra Talluri, and Alexandru Iosup. 2023 · 2023

Earlier work this paper cites.

GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

Original

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023 · 2023

Earlier work this paper cites.

Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs

Original

Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. 2023 · 2023

Cited alongside, same era.

The False Promise of Imitating Proprietary LLMs

Original

Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023 · 2023

Cited alongside, same era.

FlashDecoding++: Faster Large Language Model Inference on GPUs

Original

Ke Hong, Guohao Dai, Jiaming Xu, Qiuli Mao, Xiuhong Li, Jun Liu, Kangdi Chen, Hanyu Dong, and Yu Wang. 2023 · 2023

Cited alongside, same era.

AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving

Original

Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E Gonzalez, et al · 2023

Cited alongside, same era.

Fast Distributed Inference Serving for Large Language Models

Original

Bingyang Wu, Yinmin Zhong, Zili Zhang, Gang Huang, Xuanzhe Liu, and Xin Jin. 2023 · 2023

Later among the works it cites.

Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning . PMLR, 38087–38099

Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023 · 2023

Later among the works it cites.

Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline

Original

Zangwei Zheng, Xiaozhe Ren, Fuzhao Xue, Yang Luo, Xin Jiang, and Yang You. 2023 · 2023

Later among the works it cites.

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool

Original

Cunchen Hu, Heyang Huang, Junhao Hu, Jiang Xu, Xusheng Chen, Tao Xie, Chenxi Wang, Sa Wang, Yungang Bao, Ninghui Sun, and Yizhou Shan. 2024 · 2024

Closest in time.