One QuantLLM for ALL: Fine-tuning Quantized LLMs Once for Efficient Deployments

Understand

Large Language Models (LLMs) have advanced rapidly but face significant memory demands.

• While quantization has shown promise for LLMs, current methods typically require lengthy training to alleviate the performance degradation from quantization loss.
• However, deploying LLMs across diverse scenarios with different resource constraints, e.g., servers and personal computers, requires repeated training per application, which amplifies the lengthy training problem.
• Given that, it is advantageous to train a once-for-all (OFA) supernet capable of yielding diverse optimal subnets for downstream applications through one-shot training.