Fetching the paper…
Reading the bibliography…
Finetuning large language models (LLMs) is essential for task adaptation, yet today's serving stacks isolate inference and finetuning on separate GPU clusters -- wasting resources and under-utilizing hardware.
Fast reverse-mode automatic differentiation using expression templates in c++
Robin J Hogan · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Neural architecture search with reinforcement learning
Barret Zoph and Quoc V Le · 2016
Earlier work this paper cites.
Clipper: A { \{ Low-Latency } \} online prediction serving system
Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J Franklin, Joseph E Gonzalez, and Ion Stoica · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Earlier work this paper cites.
{ \{ PRETZEL } \} : Opening the black box of machine learning prediction serving systems
Yunseong Lee, Alberto Scolari, Byung-Gon Chun, Marco Domenico Santambrogio, Markus Weimer, and Matteo Interlandi · 2018
Earlier work this paper cites.
Gandiva: Introspective cluster scheduling for deep learning
Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, et al · 2018
Earlier work this paper cites.
Parameter-efficient transfer learning for nlp
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
Nexus: A gpu cluster engine for accelerating dnn-based video analysis
Haichen Shen, Lequn Chen, Yuchen Jin, Liangyu Zhao, Bingyu Kong, Matthai Philipose, Arvind Krishnamurthy, and Ravi Sundaram · 2019
Earlier work this paper cites.
Pretraining-based natural language generation for text summarization, 2019
Haoyu Zhang, Jianjun Xu, and Ji Wang · 2019
Earlier work this paper cites.
{ \{ PipeSwitch } \} : Fast pipelined context switching for deep learning applications
Zhihao Bai, Zhen Zhang, Yibo Zhu, and Xin Jin · 2020
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Prema: A predictive multi-task scheduling algorithm for preemptible neural processing units
Yujeong Choi and Minsoo Rhu · 2020
Earlier work this paper cites.
Gslice: controlled spatial sharing of gpus for a scalable inference platform
Aditya Dhakal, Sameer G Kulkarni, and KK Ramakrishnan · 2020
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020
Earlier work this paper cites.
Serving DNNs like clockwork: Performance predictability from the bottom up
Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, and Jonathan Mace · 2020
Earlier work this paper cites.
Checkmate: Breaking the memory wall with optimal tensor rematerialization
Paras Jain, Ajay Jain, Aniruddha Nrusimha, Amir Gholami, Pieter Abbeel, Joseph Gonzalez, Kurt Keutzer, and Ion Stoica · 2020
Earlier work this paper cites.
Adapterfusion: Non-destructive task composition for transfer learning
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych · 2020
Earlier work this paper cites.
Learning to summarize with human feedback
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano · 2020
Earlier work this paper cites.
Towards a unified view of parameter-efficient transfer learning
Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig · 2021
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Earlier work this paper cites.
The power of scale for parameter-efficient prompt tuning
Brian Lester, Rami Al-Rfou, and Noah Constant · 2021
Earlier work this paper cites.
Prefix-tuning: Optimizing continuous prompts for generation
Xiang Lisa Li and Percy Liang · 2021
Earlier work this paper cites.
What makes good in-context examples for gpt- 3 3 ?
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen · 2021
Earlier work this paper cites.
Unipelt: A unified framework for parameter-efficient language model tuning
Yuning Mao, Lambert Mathias, Rui Hou, Amjad Almahairi, Hao Ma, Jiawei Han, Wen-tau Yih, and Madian Khabsa · 2021
Earlier work this paper cites.
{ \{ INFaaS } \} : Automated model-less inference serving
Francisco Romero, Qian Li, Neeraja J Yadwadkar, and Christos Kozyrakis · 2021
Earlier work this paper cites.
Share or not? learning to schedule language-specific capacity for multilingual translation
Biao Zhang, Ankur Bapna, Rico Sennrich, and Orhan Firat · 2021
Earlier work this paper cites.
Optimizing inference serving on serverless platforms
Ahsan Ali, Riccardo Pinciroli, Feng Yan, and Evgenia Smirni · 2022
Earlier work this paper cites.
Pathways: Asynchronous distributed dataflow for ml
Paul Barham, Aakanksha Chowdhery, Jeff Dean, Sanjay Ghemawat, Steven Hand, Daniel Hurt, Michael Isard, Hyeontaek Lim, Ruoming Pang, Sudip Roy, et al · 2022
Earlier work this paper cites.
DeepSpeed-MII, 2022
DeepSpeed Team · 2022
Earlier work this paper cites.
Deep learning workload scheduling in gpu datacenters: Taxonomy, challenges and vision
Wei Gao, Qinghao Hu, Zhisheng Ye, Peng Sun, Xiaolin Wang, Yingwei Luo, Tianwei Zhang, and Yonggang Wen · 2022
Earlier work this paper cites.
Cocktail: A multidimensional optimization for model serving in cloud
Jashwant Raj Gunasekaran, Cyan Subhra Mishra, Prashanth Thinakaran, Bikash Sharma, Mahmut Taylan Kandemir, and Chita R Das · 2022
Earlier work this paper cites.
Reducing activation recomputation in large transformer models, 2022
Vijay Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Andersch, Mohammad Shoeybi, and Bryan Catanzaro · 2022
Earlier work this paper cites.
Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel · 2022
Earlier work this paper cites.
Peft: State-of-the-art parameter-efficient fine-tuning methods
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan · 2022
Earlier work this paper cites.
Learning to summarize from human feedback, 2022
Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano · 2022
Cited alongside, same era.
Unity: Accelerating DNN training through joint optimization of algebraic transformations and parallelization
Colin Unger, Zhihao Jia, Wei Wu, Sina Lin, Mandeep Baines, Carlos Efrain Quintero Narvaez, Vinay Ramakrishnaiah, Nirmal Prajapati, Patrick S. McCormick, Jamaludin Mohd-Yusof, Xi Luo, Dheevatsa Mudigere, Jongsoo Park, Misha Smelyanskiy, and Alex Aiken · 2022
Cited alongside, same era.
Transformers: State-of-the-art machine learning for pytorch, tensorflow, and jax
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush · 2022
Cited alongside, same era.
A survey of multi-tenant deep learning inference on gpu
Fuxun Yu, Di Wang, Longfei Shangguan, Minjia Zhang, Chenchen Liu, and Xiang Chen · 2022
Cited alongside, same era.
Punica: Multi-tenant lora serving
Lequn Chen, Zihao Ye, Yongji Wu, Danyang Zhuo, Luis Ceze, and Arvind Krishnamurthy · 2024
Closest in time.
Finetuning api in databricks mosaic ai training
Databricks · 2024
Closest in time.
Fine-tuning with the gemini api
Google · 2024
Closest in time.
The llama 3 herd of models, 2024
Aaron Grattafiori et al · 2024
Closest in time.
Chameleon: Adaptive caching and scheduling for many-adapter llm inference environments
Nikoleta Iliakopoulou, Jovan Stojkovic, Chloe Alverti, Tianyin Xu, Hubertus Franke, and Josep Torrellas · 2024
Closest in time.
Customizing large language model generation style using parameter-efficient finetuning
Xinyue Liu, Harshita Diddee, and Daphne Ippolito · 2024
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Orca: A distributed serving system for { \{ Transformer-Based } \} generative models
Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun · 2022
Cited alongside, same era.
Opt: Open pre-trained transformer language models
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al · 2022
Cited alongside, same era.
Alpa: Automating inter- and intra-operator parallelism for distributed deep learning
Lianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Joseph E. Gonzalez, and Ion Stoica · 2022
Cited alongside, same era.
{ \{ PetS } \} : A unified framework for { \{ Parameter-Efficient } \} transformers serving
Zhe Zhou, Xuechao Wei, Jiejing Zhang, and Guangyu Sun · 2022
Cited alongside, same era.
https://huggingface.co/models , 2023
Huggingface Models · 2023
Cited alongside, same era.
https://docs.nvidia.com/datacenter/tesla/mig-user-guide/index.html##partitioning , 2023
NVIDIA MIG Partitioning Limitations and Resetting · 2023
Cited alongside, same era.
https://www.nvidia.com/en-us/technologies/multi-instance-gpu/ , 2023
NVIDIA Multi-Instance GPU (MIG) · 2023
Cited alongside, same era.
https://docs.nvidia.com/deploy/mps/index.html , 2023
NVIDIA Multi-Process Service (MPS) · 2023
Cited alongside, same era.
Spotserve: Serving generative large language models on preemptible instances
Xupeng Miao, Chunan Shi, Jiangfei Duan, Xiaoli Xi, Dahua Lin, Bin Cui, and Zhihao Jia · 2024
Closest in time.
Fine-tuning now available for gpt-4o
OpenAI · 2024
Closest in time.
Learning to reason with llms, 2024
OpenAI · 2024
Closest in time.
Slora: Scalable serving of thousands of lora adapters
Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, et al · 2024
Closest in time.
Fairness in serving large language models, 2024
Ying Sheng, Shiyi Cao, Dacheng Li, Banghua Zhu, Zhuohan Li, Danyang Zhuo, Joseph E. Gonzalez, and Ion Stoica · 2024
Closest in time.
Democratizing large language models via personalized parameter-efficient fine-tuning
Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang · 2024
Closest in time.
Burstgpt: A real-world workload dataset to optimize llm serving systems, 2024
Yuxin Wang, Yuhan Chen, Zeyu Li, Xueze Kang, Zhenheng Tang, Xin He, Rui Guo, Xin Wang, Qiang Wang, Amelie Chi Zhou, and Xiaowen Chu · 2024
Closest in time.
Towards efficient and reliable llm serving: A real-world workload study
Yuxin Wang, Yuhan Chen, Zeyu Li, Zhenheng Tang, Rui Guo, Xin Wang, Qiang Wang, Amelie Chi Zhou, and Xiaowen Chu · 2024
Closest in time.
Lora land: 310 fine-tuned llms that rival gpt-4, a technical report
Justin Zhao, Timothy Wang, Wael Abid, Geoffrey Angus, Arnav Garg, Jeffery Kinnison, Alex Sherstinsky, Piero Molino, Travis Addair, and Devvret Rishi · 2024
Closest in time.
Sglang: Efficient execution of structured language model programs
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng · 2024
Closest in time.
Llamafactory: Unified efficient fine-tuning of 100+ language models
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma · 2024
Closest in time.
Distserve: Disaggregating prefill and decoding for goodput-optimized large language model serving, 2024
Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, and Hao Zhang · 2024
Closest in time.
Nanoflow: Towards optimal large language model serving throughput, 2024
Kan Zhu, Yilong Zhao, Liangyu Zhao, Gefei Zuo, Yile Gu, Dedong Xie, Yufei Gao, Qinyu Xu, Tian Tang, Zihao Ye, Keisuke Kamahori, Chien-Yu Lin, Stephanie Wang, Arvind Krishnamurthy, and Baris Kasikci · 2024
Closest in time.
Lithos: An operating system for efficient machine learning on gpus, 2025
Patrick H. Coppock, Brian Zhang, Eliot H. Solomon, Vasilis Kypriotis, Leon Yang, Bikash Sharma, Dan Schatzberg, Todd C. Mowry, and Dimitrios Skarlatos · 2025
Closest in time.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al · 2025
Closest in time.
Aqua: Network-accelerated memory offloading for llms in scale-up gpu domains, 2025
Abhishek Vijaya Kumar, Gianni Antichi, and Rachee Singh · 2025
Closest in time.
Optimizing safe and aligned language generation: A multi-objective grpo approach, 2025
Xuying Li, Zhuo Li, Yuji Kosuga, and Victor Bian · 2025
Closest in time.
Adaserve: Accelerating multi-slo llm serving with slo-customized speculative decoding, 2025
Zikun Li, Zhuofu Chen, Remi Delacourt, Gabriele Oliaro, Zeyu Wang, Qinghan Chen, Shuhuai Lin, April Yang, Zhihao Zhang, Zhuoming Chen, Sean Lai, Xinhao Cheng, Xupeng Miao, and Zhihao Jia · 2025
Closest in time.
Turbospec: Closed-loop speculation control system for optimizing llm serving goodput, 2025
Xiaoxuan Liu, Jongseok Park, Langxiang Hu, Woosuk Kwon, Zhuohan Li, Chen Zhang, Kuntai Du, Xiangxi Mo, Kaichao You, Alvin Cheung, Zhijie Deng, Ion Stoica, and Hao Zhang · 2025
Closest in time.
The llama 4 herd: Natively multimodal ai innovation, April 2025
Meta AI · 2025
Closest in time.
Sky-T1-32B-Preview
NovaSky-AI · 2025
Closest in time.
Sky-T1_data_17k
NovaSky-AI · 2025
Closest in time.
Suffixdecoding: Extreme speculative decoding for emerging ai applications, 2025
Gabriele Oliaro, Zhihao Jia, Daniel Campos, and Aurick Qiao · 2025
Closest in time.
Gpt-5 system card
OpenAI · 2025
Closest in time.
Hybridflow: A flexible and efficient rlhf framework
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu · 2025
Closest in time.
Pipellm: Fast and confidential large language model services with speculative pipelined encryption
Yifan Tan, Cheng Tan, Zeyu Mi, and Haibo Chen · 2025
Closest in time.
Qwen2.5 technical report, 2025
An Yang et al · 2025
Closest in time.
Qwen3 technical report, 2025
An Yang et al · 2025
Closest in time.
Dapo: An open-source llm reinforcement learning system at scale, 2025
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang · 2025
Closest in time.
Medusa: Accelerating serverless llm inference with materialization
Shaoxun Zeng, Minhui Xie, Shiwei Gao, Youmin Chen, and Youyou Lu · 2025
Closest in time.