Fetching the paper…
Reading the bibliography…
The recent rise of large language models (LLMs) has resulted in increased efforts towards running LLMs at reduced precision.
Qpytorch: A low-precision arithmetic simulation framework
Tianyi Zhang, Zhiqiu Lin, Guandao Yang, and Christopher De Sa · 2019
Earlier work this paper cites.
Integer quantization for deep learning inference: Principles and empirical evaluation
Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius · 2020
Earlier work this paper cites.
Vs-quant: Per-vector scaled quantization for accurate low-precision neural network inference
Steve Dai, Rangha Venkatesan, Mark Ren, Brian Zimmer, William Dally, and Brucek Khailany · 2021
Earlier work this paper cites.
An electro-photonic system for accelerating deep neural networks
Cansu Demirkiran, Furkan Eris, Gongyu Wang, Jonathan Elmhurst, Nick Moore, Nicholas C Harris, Ayon Basumallik, Vijay Janapa Reddi, Ajay Joshi, and Darius Bunandar · 2021
Earlier work this paper cites.
Smoothquant: Accurate and efficient post-training quantization for large language models
Guangxuan Xiao, Ji Lin, Mickael Seznec, Julien Demouth, and Song Han · 2022
Earlier work this paper cites.
Llm. int8 (): 8-bit matrix multiplication for transformers at scale
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer · 2022
Cited alongside, same era.
Gptq: Accurate post-training quantization for generative pre-trained transformers
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh · 2022
Cited alongside, same era.
Adaptive block floating-point for analog deep learning hardware
Ayon Basumallik, Darius Bunandar, Nicholas Dronen, Nicholas Harris, Ludmila Levkova, Calvin McCarter, Lakshmi Nair, David Walter, and David Widemann · 2022
Cited alongside, same era.
Fp8 quantization: The power of the exponent
Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, and Tijmen Blankevoort · 2022
Cited alongside, same era.
Optimal clipping and magnitude-aware differentiation for improved quantization-aware training
Charbel Sakr, Steve Dai, Rangha Venkatesan, Brian Zimmer, William Dally, and Brucek Khailany · 2022
Later among the works it cites.
Rptq: Reorder-based post-training quantization for large language models
Zhihang Yuan, Lin Niu, Jiawei Liu, Wenyu Liu, Xinggang Wang, Yuzhang Shang, Guangyu Sun, Qiang Wu, Jiaxiang Wu, and Bingzhe Wu · 2023
Closest in time.
https://github.com/NVIDIA/TensorRT/tree/master/tools/pytorch-quantization
Tensorrt - pytorch quantization · 2023
Closest in time.
https://github.com/quic/aimet
Ai model efficiency toolkit · 2023
Closest in time.
A comprehensive study on post-training quantization for large language models
Zhewei Yao, Cheng Li, Xiaoxia Wu, Stephen Youn, and Yuxiong He · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, et al · 2022
Cited alongside, same era.