Personalized Preference Fine-tuning of Diffusion Models

Understand

RLHF techniques like DPO can significantly improve the generation quality of text-to-image diffusion models.

• However, these methods optimize for a single reward that aligns model generation with population-level preferences, neglecting the nuances of individual users' beliefs or values.
• This lack of personalization limits the efficacy of these models.
• To bridge this gap, we introduce PPD, a multi-reward optimization objective that aligns diffusion models with personalized preferences.