Null-text Guidance in Diffusion Models is Secretly a Cartoon-style Creator

Understand

Classifier-free guidance is an effective sampling technique in diffusion models that has been widely adopted.

• The main idea is to extrapolate the model in the direction of text guidance and away from null-text guidance.
• In this paper, we demonstrate that null-text guidance in diffusion models is secretly a cartoon-style creator, i.e., the generated images can be efficiently transformed into cartoons by simply perturbing the null-text guidance.
• Specifically, we proposed two disturbance methods, i.e., Rollback disturbance (Back-D) and Image disturbance (Image-D), to construct misalignment between the noisy images used for predicting null-text guidance and text guidance (subsequently referred to as \textbf{null-text noisy image} and \textbf{text noisy image} respectively) in the sampling process.