Fetching the paper…
Reading the bibliography…
The complexity of large language model (LLM) serving workloads has substantially increased due to the integration with external tool invocations, such as ChatGPT plugins.
Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala · 2019
Earlier work this paper cites.
DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He · 2020
Earlier work this paper cites.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré · 2022
Earlier work this paper cites.
Orca: A Distributed Serving System for Transformer-Based Generative Models
Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun · 2022
Earlier work this paper cites.
Learning and Leveraging Verifiers to Improve Planning Capabilities of Pre-trained Language Models
Daman Arora and Subbarao Kambhampati · 2023
Earlier work this paper cites.
Accelerating Large Language Model Decoding with Speculative Sampling
Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper · 2023
Earlier work this paper cites.
Punica: Multi-Tenant LoRA Serving
Lequn Chen, Zihao Ye, Yongji Wu, Danyang Zhuo, Luis Ceze, and Arvind Krishnamurthy · 2023
Earlier work this paper cites.
An LLM Compiler for Parallel Function Calling
Sehoon Kim, Suhong Moon, Ryan Tabrizi, Nicholas Lee, Michael W. Mahoney, Kurt Keutzer, and Amir Gholami · 2023
Earlier work this paper cites.
Validating Large Language Models with ReLM
Michael Kuchnik, Virginia Smith, and George Amvrosiadis · 2023
Earlier work this paper cites.
Efficient Memory Management for Large Language Model Serving with PagedAttention
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica · 2023
Cited alongside, same era.
Fast Inference from Transformers via Speculative Decoding
Yaniv Leviathan, Matan Kalman, and Yossi Matias · 2023
Cited alongside, same era.
TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage, November 2023
Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng, and Rui Zhao · 2023
Cited alongside, same era.
Toolformer: Language Models Can Teach Themselves to Use Tools
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom · 2023
Cited alongside, same era.
ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models, May 2023
Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, and Dongkuan Xu · 2023
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang · 2024
Closest in time.
https://github.com/MeetKai/functionary , 2024
Functionary is a language model that can interpret and execute functions/plugins · 2024
Closest in time.
Bin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang, Djordje Jevdjic, Junbo Deng, Xingkun Yang, Zhou Yu, and Pengfei Zuo · 2024
Closest in time.
Adaptive Skeleton Graph Decoding
Shuowei Jin, Yongji Wu, Haizhong Zheng, Qingzhao Zhang, Matthew Lentz, Z. Morley Mao, Atul Prakash, Feng Qian, and Danyang Zhuo · 2024
Closest in time.
https://www.kayak.com/news/kayak-chatgpt/ , 2024
Welcome, robots: KAYAK is now integrated on ChatGPT · 2024
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
InferCept: Efficient Intercept Support for Augmented Large Language Model Inference
Reyna Abhyankar, Zijian He, Vikranth Srivatsa, Hao Zhang, and Yiying Zhang · 2024
Cited alongside, same era.
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao · 2024
Cited alongside, same era.
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
Tri Dao · 2024
Cited alongside, same era.
https://github.com/flashinfer-ai/flashinfer , 2024
FlashInfer: Kernel Library for LLM Serving · 2024
Cited alongside, same era.
https://www.langchain.com/ , 2024
Applications that can reason. Powered by LangChain · 2024
Closest in time.
Optimizing LLM Queries in Relational Workloads, March 2024
Shu Liu, Asim Biswal, Audrey Cheng, Xiangxi Mo, Shiyi Cao, Joseph E. Gonzalez, Ion Stoica, and Matei Zaharia · 2024
Closest in time.
https://mistral.ai/news/announcing-mistral-7b/ , 2024
Mistral AI team is proud to release Mistral 7B, the most powerful language model for its size to date · 2024
Closest in time.
https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/ , 2024
ChatGPT Gets Its “Wolfram Superpowers”! · 2024
Closest in time.