Multi-Reward as Condition for Instruction-based Image Editing

Understand

High-quality training triplets (instruction, original image, edited image) are essential for instruction-based image editing.

• Predominant training datasets (e.g., InsPix2Pix) are created using text-to-image generative models (e.g., Stable Diffusion, DALL-E) which are not trained for image editing.
• Accordingly, these datasets suffer from inaccurate instruction following, poor detail preserving, and generation artifacts.
• In this paper, we propose to address the training data quality issue with multi-perspective reward data instead of refining the ground-truth image quality.