A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms

Understand

Large language models (LLMs) have achieved remarkable advancements in natural language processing, showcasing exceptional performance across various tasks.

• However, the expensive memory and computational requirements present significant challenges for their practical deployment.
• Low-bit quantization has emerged as a critical approach to mitigate these challenges by reducing the bit-width of model parameters, activations, and gradients, thus decreasing memory usage and computational demands.
• This paper presents a comprehensive survey of low-bit quantization methods tailored for LLMs, covering the fundamental principles, system implementations, and algorithmic strategies.