Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study

Understand

Despite the superior performance, Large Language Models~(LLMs) require significant computational resources for deployment and use.

• To overcome this issue, quantization methods have been widely applied to reduce the memory footprint of LLMs as well as increasing the inference rate.
• However, a major challenge is that low-bit quantization methods often lead to performance degradation.
• It is important to understand how quantization impacts the capacity of LLMs.