VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation

Understand

Visual generative models have achieved remarkable progress in synthesizing photorealistic images and videos, yet aligning their outputs with human preferences across critical dimensions remains a persistent challenge.

• Though reinforcement learning from human feedback offers promise for preference alignment, existing reward models for visual generation face limitations, including black-box scoring without interpretability and potentially resultant unexpected biases.
• We present VisionReward, a general framework for learning human visual preferences in both image and video generation.
• Specifically, we employ a hierarchical visual assessment framework to capture fine-grained human preferences, and leverages linear weighting to enable interpretable preference learning.