Fetching the paper…

Reading the bibliography…

2024

LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs

Xu, Han, Li, Yutong, Ji, Shihao

Original
PDF
Post on X

Understand

Large language models (LLMs) have demonstrated remarkable abilities in natural language processing.

However, their deployment on resource-constrained embedded devices remains difficult due to memory and computational demands.
In this paper, we present an FPGA-based accelerator designed to improve LLM inference performance on embedded FPGAs.
We employ post-training quantization to reduce model size and optimize for off-chip memory bandwidth.

LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs · Around

Built on

P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ Questions for Machine Comprehension of Text,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , 2016
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention Is All You Need,” in Advances in Neural Information Processing Systems , 2017
2017
Earlier work this paper cites.
S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer Sentinel Mixture Models,” in International Conference on Learning Representations , 2017
2017
Earlier work this paper cites.
B. Zhang and R. Sennrich, “Root Mean Square Layer Normalization,” in Advances in Neural Information Processing Systems , 2019
2019
Earlier work this paper cites.
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , 2019
2019
Earlier work this paper cites.
A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, “The Curious Case of Neural Text Degeneration,” in International Conference on Learning Representations , 2020
2020
Earlier work this paper cites.
N. Shazeer, “GLU Variants Improve Transformer,” arXiv preprint arXiv:2002.05202, 2020
Original
2020
Earlier work this paper cites.
S. Shen, Z. Dong, J. Ye, L. Ma, Z. Yao, A. Gholami, M. W. Mahoney, and K. Keutzer, “Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT,” in Proceedings of the AAAI Conference on Artificial Intelligence , 2020
2020
Earlier work this paper cites.
T. B. Brown et al., “Language Models are Few-Shot Learners,” in Advances in Neural Information Processing Systems , 2020
2020
Earlier work this paper cites.
A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A Survey of Quantization Methods for Efficient Neural Network Inference,” arXiv preprint arXiv:2103.13630, 2021
Original
2021
Earlier work this paper cites.

Similar

Z. Liu, G. Li, and J. Cheng, “Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processing,” in Proceedings of the 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE) , 2021
2021
Cited alongside, same era.
S. Zhang, et al., “OPT: Open Pre-trained Transformer Language Models,” arXiv preprint arXiv:2205.01068, 2022
Original
2022
Cited alongside, same era.
C. Peng, X. Yang, A. Chen, et al., “A Study of Generative Large Language Model for Medical Research and Healthcare,” npj Digital Medicine, vol. 6, no. 210, 2023
2023
Cited alongside, same era.
F. Zeng, W. Gan, Y. Wang, N. Liu, and P. S. Yu, “Large Language Models for Robotics: A Survey,” arXiv preprint arXiv:2311.07226, 2023
Original
2023
Cited alongside, same era.
H. Touvron et al., “Llama 2: Open Foundation and Fine-Tuned Chat Models,” arXiv preprint arXiv:2307.09288, 2023

Then

W. Liu, Z. He, and X. Huang, “Time Matters: Examine Temporal Effects on Biomedical Language Models,” arXiv preprint arXiv:2407.17638, 2024
Original
2024
Closest in time.
Z. Ding, P. Li, Q. Yang, and S. Li, “Enhance Image-to-Image Generation with LLaVA Prompt and Negative Prompt,” arXiv preprint arXiv:2406.01956, 2024
Original
2024
Closest in time.
Q. Deng, et al., “ComposerX: Multi-Agent Symbolic Music Composition with LLMs,” arXiv preprint arXiv:2404.18081, 2024
Original
2024
Closest in time.
Q. Pan, H. Cao, Y. Zhu, J. Liu, and B. Li, “Contextual Client Selection for Efficient Federated Learning Over Edge Devices,” IEEE Transactions on Mobile Computing , vol. 23, no. 6, pp. 6538–6548, 2024
2024
Closest in time.
P. Zhang, G. Zeng, T. Wang, and W. Lu, “TinyLlama: An Open-Source Small Language Model,” arXiv preprint arXiv:2401.02385, 2024

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2023

Cited alongside, same era.

X. Liu et al. , “Influence Pathway Discovery on Social Media,” in 2023 IEEE 9th International Conference on Collaboration and Internet Computing (CIC) , 2023

2023

Cited alongside, same era.

S. Li, R. Zhao, M. Li, H. Ji, C. Callison-Burch, and J. Han, “Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023

2023

Cited alongside, same era.

Y. Liu, J. Huang, and K. Chang, “Ask To The Point: Open-Domain Entity-Centric Question Generation,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023

2023

Cited alongside, same era.

J. Ainslie, J. Lee-Thorp, M. de Jong, Y. Zemlyanskiy, F. Lebron, and S. Sanghai, “GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , 2023

2023

Cited alongside, same era.

J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu, “RoFormer: Enhanced Transformer with Rotary Position Embedding,” arXiv preprint arXiv:2104.09864, 2023

2023

Cited alongside, same era.

2024

Closest in time.

K. Mo, W. Liu, X. Xu, C. Yu, Y. Zou, and F. Xia, “Fine-Tuning Gemma-7B for Enhanced Sentiment Analysis of Financial News Headlines,” arXiv preprint arXiv:2406.13626, 2024

2024

Closest in time.

B. Jiang, Z. Zhuang, S. S. Shivakumar, D. Roth, and C. J. Taylor, “Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answering,” arXiv preprint arXiv:2403.14783, 2024

2024

Closest in time.

H. Chen, J. Zhang, Y. Du, S. Xiang, Z. Yue, N. Zhang, Y. Cai, and Z. Zhang, “Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference,” ACM Transactions on Reconfigurable Technology and Systems , 2024

2024

Closest in time.

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language Models are Unsupervised Multitask Learners,” OpenAI blog. [Online]. Available: https://openai.com/research/better-language-models . Accessed: Mar. 01, 2024

2024

Closest in time.

S. Zeng et al., “FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs,” in Proceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays , 2024

2024

Closest in time.