Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models

Vector quantization

Robert Gray. 1984 · 1984

Earlier work this paper cites.

Quantization

Robert M. Gray and David L. Neuhoff. 1998 · 1998

Earlier work this paper cites.

HotpotQA: A dataset for diverse, explainable multi-hop question answering

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018 · 2018

Earlier work this paper cites.

Transformers: State-of-the-art natural language processing

Thomas Wolf et al. 2020 · 2020

Earlier work this paper cites.

Optimal brain compression: A framework for accurate post-training quantization and pruning

Elias Frantar and Dan Alistarh. 2022 · 2022

Earlier work this paper cites.

Gptq: Accurate post-training quantization for generative pre-trained transformers

Original

Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022 · 2022

Earlier work this paper cites.

Training compute-optimal large language models

Original

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. 2022 · 2022

Earlier work this paper cites.

Language models (mostly) know what they know

Original

Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. 2022 · 2022

Earlier work this paper cites.

SafeText: A benchmark for exploring physical safety in language models

Sharon Levy, Emily Allaway, Melanie Subbiah, Lydia Chilton, Desmond Patton, Kathleen McKeown, and William Yang Wang. 2022 · 2022

Earlier work this paper cites.

Towards end-to-end 4-bit inference on generative large language models

Original

Saleh Ashkboos, Ilia Markov, Elias Frantar, Tingxuan Zhong, Xincheng Wang, Jie Ren, Torsten Hoefler, and Dan Alistarh. 2023 · 2023

Earlier work this paper cites.

Mistral 7b

Original

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023 · 2023

Earlier work this paper cites.

vllm: Easy, fast, and memory-efficient llm serving

Woosuk Kwon et al. 2023 · 2023

Earlier work this paper cites.

Direct preference optimization: Your language model is secretly a reward model

Original

Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023 · 2023

Earlier work this paper cites.

Safety assessment of chinese large language models

Original

Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng, and Minlie Huang. 2023 · 2023

Cited alongside, same era.

Smoothquant: Accurate and efficient post-training quantization for large language models

Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023 · 2023

Cited alongside, same era.

AlignScore: Evaluating factual consistency with a unified alignment function

Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. 2023 · 2023

Cited alongside, same era.

Safetybench: Evaluating the safety of large language models with multiple choice questions

Original

Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. 2023 · 2023

Cited alongside, same era.

Refusal in language models is mediated by a single direction

Original

A comprehensive evaluation of quantization strategies for large language models

Original

Renren Jin, Jiangcun Du, Wuwei Huang, Wei Liu, Jian Luan, Bin Wang, and Deyi Xiong. 2024 · 2024

Later among the works it cites.

Awq: Activation-aware weight quantization for on-device llm compression and acceleration

Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024 · 2024

Later among the works it cites.

Evaluating the generalization ability of quantized llms: Benchmark, analysis, and toolbox

Original

Yijun Liu, Yuan Meng, Fang Wu, Shenhao Peng, Hang Yao, Chaoyu Guan, Chen Tang, Xinzhu Ma, Zhi Wang, and Wenwu Zhu. 2024 · 2024

Later among the works it cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2024 · 2024

Later among the works it cites.