Fast Inference for Augmented Large Language Models

Modern information retrieval . Vol. 463

Ricardo Baeza-Yates, Berthier Ribeiro-Neto, et al · 1999

Earlier work this paper cites.

AgentGraph: Toward universal dialogue management with structured deep reinforcement learning

Lu Chen, Zhi Chen, Bowen Tan, Sishan Long, Milica Gašić, and Kai Yu. 2019 · 2019

Earlier work this paper cites.

Alfworld: Aligning text and embodied environments for interactive learning

Original

Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2020 · 2020

Earlier work this paper cites.

Queues with Small Advice. In Proceedings of the 2021 SIAM Conference on Applied and Computational Discrete Algorithms (ACDA 21) . 1–12

Michael Mitzenmacher. 2021 · 2021

Earlier work this paper cites.

Introducing ChatGPT

OpenAI. 2022 · 2022

Earlier work this paper cites.

Orca: A distributed serving system for { \{ Transformer-Based } \} generative models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) . 521–538

Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022 · 2022

Earlier work this paper cites.

Opt: Open pre-trained transformer language models

Original

Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al · 2022

Earlier work this paper cites.

Improving image generation with better captions

James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al · 2023

Earlier work this paper cites.

$S^3$: Increasing GPU Utilization during Generative Inference for Higher Throughput. In Thirty-seventh Conference on Neural Information Processing Systems

Yunho Jin, Chun-Feng Wu, David Brooks, and Gu-Yeon Wei. 2023 · 2023

Earlier work this paper cites.

Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles . 611–626

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023 · 2023

Earlier work this paper cites.

Augmented language models: a survey

Original

Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, et al · 2023

Cited alongside, same era.

ChatGPT plugins

OpenAI. March 2023 · 2023

Cited alongside, same era.

Gorilla: Large language model connected with massive apis

Original

Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2023 · 2023

Cited alongside, same era.

ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

Original

Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023 · 2023

Cited alongside, same era.

Chatgpt gets its ‘wolfram superpowers

Wolfram. [n.d.] · 2023

Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings

Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. 2024 · 2024

Closest in time.

DSPy: Compiling Declarative Language Model Calls into State-of-the-Art Pipelines. In The Twelfth International Conference on Learning Representations

Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Saiful Haq, Ashutosh Sharma, Thomas T Joshi, Hanna Moazam, Heather Miller, et al · 2024

Closest in time.

Splitwise: Efficient generative llm inference using phase splitting. In 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) . IEEE, 118–132

Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, and Ricardo Bianchini. 2024 · 2024

Closest in time.

Power-aware Deep Learning Model Serving with { \{ μ \mu -Serve } \} . In 2024 USENIX Annual Technical Conference (USENIX ATC 24) . 75–93

Haoran Qiu, Weichao Mao, Archit Patke, Shengkun Cui, Saurabh Jha, Chen Wang, Hubertus Franke, Zbigniew Kalbarczyk, Tamer Başar, and Ravishankar K Iyer. 2024a · 2024

Closest in time.