Inference-Time Alignment of Diffusion Models with Direct Noise Optimization

Understand

In this work, we focus on the alignment problem of diffusion models with a continuous reward function, which represents specific objectives for downstream tasks, such as increasing darkness or improving the aesthetics of images.

• The central goal of the alignment problem is to adjust the distribution learned by diffusion models such that the generated samples maximize the target reward function.
• We propose a novel alignment approach, named Direct Noise Optimization (DNO), that optimizes the injected noise during the sampling process of diffusion models.
• By design, DNO operates at inference-time, and thus is tuning-free and prompt-agnostic, with the alignment occurring in an online fashion during generation.