DeltaLLM: A Training-Free Framework Exploiting Temporal Sparsity for Efficient Edge LLM Inference

C. Clark, K. Lee, M.-W. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova, “Boolq: Exploring the surprising difficulty of natural yes/no questions,” 2019. [Online]. Available: https://arxiv.org/abs/1905.10044

Original

1905

Earlier work this paper cites.

R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi, “Hellaswag: Can a machine really finish your sentence?” 2019. [Online]. Available: https://arxiv.org/abs/1905.07830

Original

1905

Earlier work this paper cites.

Y. Bisk, R. Zellers, R. L. Bras, J. Gao, and Y. Choi, “Piqa: Reasoning about physical commonsense in natural language,” 2019. [Online]. Available: https://arxiv.org/abs/1911.11641

Original

1911

Earlier work this paper cites.

D. Neil, J. H. Lee, T. Delbruck, and S.-C. Liu, “Delta networks for optimized recurrent network computation,” in International conference on machine learning . PMLR, 2017, pp. 2584–2593

2017

Earlier work this paper cites.

C. Gao, D. Neil, E. Ceolini, S.-C. Liu, and T. Delbruck, “Deltarnn: A power-efficient recurrent neural network accelerator,” in Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays , 2018, pp. 21–30

2018

Earlier work this paper cites.

P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the ai2 reasoning challenge,” arXiv:1803.05457v1 , 2018

Original

2018

Earlier work this paper cites.

T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal, “Can a suit of armor conduct electricity? a new dataset for open book question answering,” in EMNLP , 2018

2018

Earlier work this paper cites.

P. Rajpurkar, J. Zhang, and P. Liang, “Know what you don’t know: Unanswerable questions for squad,” in ACL 2018 , 2018

2018

Earlier work this paper cites.

“Winogrande: An adversarial winograd schema challenge at scale,” 2019

2019

Earlier work this paper cites.

C. Gao, A. Rios-Navarro, X. Chen, S.-C. Liu, and T. Delbruck, “Edgedrnn: Recurrent neural network accelerator for edge inference,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems , vol. 10, no. 4, pp. 419–432, 2020

2020

Earlier work this paper cites.

A. Habibian, D. Abati, T. S. Cohen, and B. E. Bejnordi, “Skip-convolutions for efficient video processing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 2695–2704

2021

Earlier work this paper cites.

C. Gao, T. Delbruck, and S.-C. Liu, “Spartus: A 9.4 top/s fpga-based lstm accelerator exploiting spatio-temporal sparsity,” IEEE Transactions on Neural Networks and Learning Systems , vol. 35, no. 1, pp. 1098–1112, 2022

2022

Cited alongside, same era.

M. Parger, C. Tang, C. D. Twigg, C. Keskin, R. Wang, and M. Steinberger, “Deltacnn: End-to-end cnn inference of sparse frame differences in videos,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022, pp. 12 487–12 496

2022

Cited alongside, same era.

Z. Jelčicová and M. Verhelst, “Delta keyword transformer: Bringing transformers to the edge through dynamically pruned multi-head self-attention,” arXiv preprint arXiv:2204.03479 , 2022

Original

2022

Cited alongside, same era.

S. Bubeck, V. Chadrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg et al. , “Sparks of artificial general intelligence: Early experiments with gpt-4,” 2023

2023

T. Cai, Y. Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, and T. Dao, “Medusa: Simple llm inference acceleration framework with multiple decoding heads,” arXiv preprint arXiv:2401.10774 , 2024

Original

2024

Later among the works it cites.

H. Xia, Z. Yang, Q. Dong, P. Wang, Y. Li, T. Ge, T. Liu, W. Li, and Z. Sui, “Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding,” arXiv preprint arXiv:2401.07851 , 2024

Original

2024

Later among the works it cites.

Y. Deng, Z. Song, and C. Yang, “Attention is naturally sparse with gaussian distributed input,” arXiv preprint arXiv:2404.02690 , 2024

Original

2024

Later among the works it cites.

H. Jiang, Y. Li, C. Zhang, Q. Wu, X. Luo, S. Ahn, Z. Han, A. H. Abdi, D. Li, C.-Y. Lin et al. , “Minference 1.0: Accelerating pre-filling for long-context llms via dynamic sparse attention,” arXiv preprint arXiv:2407.02490 , 2024

Original

2024

Later among the works it cites.