SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , Language models are few-shot learners, Advances in neural information processing systems , 2020, 33

1901

Earlier work this paper cites.

Y. Bengio, N. Léonard and A. Courville, Estimating or propagating gradients through stochastic neurons for conditional computation, arXiv preprint arXiv:1308.3432 , 2013

Original

2013

Earlier work this paper cites.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser and I. Polosukhin, Attention is all you need, Advances in neural information processing systems , 2017, 30

2017

Earlier work this paper cites.

B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam and D. Kalenichenko, Quantization and training of neural networks for efficient integer-arithmetic-only inference , Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2704–2713

2018

Earlier work this paper cites.

J. Choi, Z. Wang, S. Venkataramani, P. I.-J. Chuang, V. Srinivasan and K. Gopalakrishnan, Pact: Parameterized clipping activation for quantized neural networks, arXiv preprint arXiv:1805.06085 , 2018

Original

2018

Earlier work this paper cites.

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , Language models are unsupervised multitask learners, OpenAI blog , 2019, 1

2019

Earlier work this paper cites.

S. K. Esser, J. L. McKinstry, D. Bablani, R. Appuswamy and D. S. Modha, Learned step size quantization, arXiv preprint arXiv:1902.08153 , 2019

Original

2019

Earlier work this paper cites.

L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima et al. , The pile: An 800gb dataset of diverse text for language modeling, arXiv preprint arXiv:2101.00027 , 2020

Original

2020

Earlier work this paper cites.

C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li and P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, The Journal of Machine Learning Research , 2020, 21

2020

Earlier work this paper cites.

H. Wu, P. Judd, X. Zhang, M. Isaev and P. Micikevicius, Integer quantization for deep learning inference: Principles and empirical evaluation, arXiv preprint arXiv:2004.09602 , 2020

Original

2020

Cited alongside, same era.

M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al. , Evaluating large language models trained on code, arXiv preprint arXiv:2107.03374 , 2021

Original

2021

Cited alongside, same era.

J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai and Q. V. Le, Finetuned language models are zero-shot learners, arXiv preprint arXiv:2109.01652 , 2021

Original

2021

Cited alongside, same era.

S. Black, S. Biderman, E. Hallahan, Q. Anthony, L. Gao, L. Golding, H. He, C. Leahy, K. McDonell, J. Phang et al. , Gpt-neox-20b: An open-source autoregressive language model, 2022, URL https://arxiv. org/abs/2204.06745

Original

2022

Cited alongside, same era.

Z. Liu, B. Oguz, C. Zhao, E. Chang, P. Stock, Y. Mehdad, Y. Shi, R. Krishnamoorthi and V. Chandra, LLM-QAT: Data-Free Quantization Aware Training for Large Language Models, arXiv preprint arXiv:2305.17888 , 2023

Original

2023

Closest in time.

J. Lin, J. Tang, H. Tang, S. Yang, X. Dang and S. Han, AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration, arXiv preprint arXiv:2306.00978 , 2023

Original

2023

Closest in time.

C. Lee, J. Jin, T. Kim, H. Kim and E. Park, OWQ: Lessons learned from activation outliers for weight quantization in large language models, arXiv preprint arXiv:2306.02272 , 2023

Original

2023

Closest in time.

B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin et al. , Code llama: Open foundation models for code, arXiv preprint arXiv:2308.12950 , 2023

Original

2023

Closest in time.