SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

Understand

Diffusion models can effectively generate high-quality images.

• However, as they scale, rising memory demands and higher latency pose substantial deployment challenges.
• In this work, we aim to accelerate diffusion models by quantizing their weights and activations to 4 bits.
• At such an aggressive level, both weights and activations are highly sensitive, where existing post-training quantization methods like smoothing become insufficient.