TIE: Revolutionizing Text-based Image Editing for Complex-Prompt Following and High-Fidelity Editing

Understand

As the field of image generation rapidly advances, traditional diffusion models and those integrated with multimodal large language models (LLMs) still encounter limitations in interpreting complex prompts and preserving image consistency pre and post-editing.

• To tackle these challenges, we present an innovative image editing framework that employs the robust Chain-of-Thought (CoT) reasoning and localizing capabilities of multimodal LLMs to aid diffusion models in generating more refined images.
• We first meticulously design a CoT process comprising instruction decomposition, region localization, and detailed description.
• Subsequently, we fine-tune the LISA model, a lightweight multimodal LLM, using the CoT process of Multimodal LLMs and the mask of the edited image.