Learning Action and Reasoning-Centric Image Editing from Videos and Simulations

Understand

An image editing model should be able to perform diverse edits, ranging from object replacement, changing attributes or style, to performing actions or movement, which require many forms of reasoning.

• Current general instruction-guided editing models have significant shortcomings with action and reasoning-centric edits.
• Object, attribute or stylistic changes can be learned from visually static datasets.
• On the other hand, high-quality data for action and reasoning-centric edits is scarce and has to come from entirely different sources that cover e.g.