FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs

P. Kurup and T. Abbasi, Logic synthesis using Synopsys® . Springer Science & Business Media, 1997

1997

Earlier work this paper cites.

S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer sentinel mixture models,” arXiv preprint arXiv:1609.07843 , 2016

Original

2016

Earlier work this paper cites.

C. H. Oliveira, M. T. Moreira, R. A. Guazzelli, and N. L. Calazans, “Ascend-freepdk45: An open source standard cell library for asynchronous design,” in 2016 IEEE International Conference on Electronics, Circuits and Systems (ICECS) . IEEE, 2016, pp. 652–655

2016

Earlier work this paper cites.

A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems , 2017

2017

Earlier work this paper cites.

V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, “Efficient processing of deep neural networks: A tutorial and survey,” Proceedings of the IEEE , vol. 105, no. 12, pp. 2295–2329, 2017

2017

Earlier work this paper cites.

H. Kwon, P. Chatarasi, M. Pellauer, A. Parashar, V. Sarkar, and T. Krishna, “Understanding reuse, performance, and hardware cost of dnn dataflow: A data-centric approach,” in Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture , 2019, pp. 754–768

2019

Earlier work this paper cites.

M. Nagel, R. A. Amjad, M. Van Baalen, C. Louizos, and T. Blankevoort, “Up or down? adaptive rounding for post-training quantization,” in International Conference on Machine Learning . PMLR, 2020, pp. 7197–7206

2020

Earlier work this paper cites.

C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research , vol. 21, no. 140, pp. 1–67, 2020

2020

Earlier work this paper cites.

D. Wu, J. Li, Z. Pan, Y. Kim, and J. S. Miguel, “ubrain: A unary brain computer interface,” in Proceedings of the 49th Annual International Symposium on Computer Architecture , 2022, pp. 468–481

2022

Earlier work this paper cites.

E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “Gptq: Accurate post-training quantization for generative pre-trained transformers,” arXiv preprint arXiv:2210.17323 , 2022

Original

2022

Earlier work this paper cites.

Z. Yao, R. Yazdani Aminabadi, M. Zhang, X. Wu, C. Li, and Y. He, “Zeroquant: Efficient and affordable post-training quantization for large-scale transformers,” Advances in Neural Information Processing Systems , vol. 35, pp. 27 168–27 183, 2022

2022

Cited alongside, same era.

T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer, “Llm. int8 (): 8-bit matrix multiplication for transformers at scale. corr abs/2208.07339 (2022),” 2022

Original

2022

Cited alongside, same era.

T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer, “Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale,” Advances in Neural Information Processing Systems , vol. 35, pp. 30 318–30 332, 2022

2022

Cited alongside, same era.

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023

Original

2023

Cited alongside, same era.

W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223 , 2023

Original

2023

Later among the works it cites.

S. Kim, C. Hooper, A. Gholami, Z. Dong, X. Li, S. Shen, M. W. Mahoney, and K. Keutzer, “Squeezellm: Dense-and-sparse quantization,” arXiv preprint arXiv:2306.07629 , 2023

Original

2023

Later among the works it cites.

X. Wei, Y. Zhang, Y. Li, X. Zhang, R. Gong, J. Guo, and X. Liu, “Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling,” arXiv preprint arXiv:2304.09145 , 2023

Original

2023

Later among the works it cites.

J. Lin, J. Tang, H. Tang, S. Yang, W.-M. Chen, W.-C. Wang, G. Xiao, X. Dang, C. Gan, and S. Han, “Awq: Activation-aware weight quantization for on-device llm compression and acceleration,” Proceedings of Machine Learning and Systems , vol. 6, pp. 87–100, 2024

2024

Later among the works it cites.