User-friendly Image Editing with Minimal Text Input: Leveraging Captioning and Injection Techniques

Understand

Recent text-driven image editing in diffusion models has shown remarkable success.

• However, the existing methods assume that the user's description sufficiently grounds the contexts in the source image, such as objects, background, style, and their relations.
• This assumption is unsuitable for real-world applications because users have to manually engineer text prompts to find optimal descriptions for different images.
• From the users' standpoint, prompt engineering is a labor-intensive process, and users prefer to provide a target word for editing instead of a full sentence.