Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in

2017

Earlier work this paper cites.

Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,”

2019

Earlier work this paper cites.

E. Li, L. Zeng, Z. Zhou, and X. Chen, “Edge AI: On-demand accelerating deep neural network inference via edge computing,”

2020

Earlier work this paper cites.

G.-I. Yu, J. S. Jeong, G.-W. Kim, S. Kim, and B.-G. Chun, “Orca: A distributed serving system for transformer-based generative models,” in

2022

Earlier work this paper cites.

V. Cacchiani, M. Iori, A. Locatelli, and S. Martello, “Knapsack problems—an overview of recent advances. part ii: Multiple, multidimensional, and quadratic knapsack problems,”

2022

Earlier work this paper cites.

P. Hacker, A. Engel, and M. Mauer, “Regulating ChatGPT and other large generative AI models,” in

2023

Cited alongside, same era.

W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong

Original

2023

Cited alongside, same era.

M. Xu, H. Du, D. Niyato, J. Kang, Z. Xiong, S. Mao, Z. Han, A. Jamalipour, D. I. Kim, V. Leung

Original

2023

Cited alongside, same era.

J. Li, W. Liang, Y. Li, Z. Xu, X. Jia, and S. Guo, “Throughput maximization of delay-aware dnn inference in edge computing by exploring dnn model partitioning and inference parallelism,”

2023

Cited alongside, same era.

F. Dong, H. Wang, D. Shen, Z. Huang, Q. He, J. Zhang, L. Wen, and T. Zhang, “Multi-exit dnn inference acceleration based on multi-dimensional optimization for edge intelligence,”

Z. Yao, C. Li, X. Wu, S. Youn, and Y. He, “A comprehensive study on post-training quantization for large language models,”

Original

2023

Later among the works it cites.

Z. Liu, Q. Lan, and K. Huang, “Resource allocation for multiuser edge inference with batching and early exiting,”

2023

Later among the works it cites.

Y. Sheng, L. Zheng, B. Yuan, Z. Li, M. Ryabinin, D. Y. Fu, Z. Xie, B. Chen, C. Barrett, J. E. Gonzalez

Original

2023

Later among the works it cites.

C. W. Zaw, N. H. Tran, Z. Han, and C. S. Hong, “Radio and computing resource allocation in co-located edge computing: A generalized nash equilibrium model,”

2023

Later among the works it cites.