Improving Cross-modal Alignment for Text-Guided Image Inpainting

Understand

Text-guided image inpainting (TGII) aims to restore missing regions based on a given text in a damaged image.

• Existing methods are based on a strong vision encoder and a cross-modal fusion model to integrate cross-modal features.
• However, these methods allocate most of the computation to visual encoding, while light computation on modeling modality interactions.
• Moreover, they take cross-modal fusion for depth features, which ignores a fine-grained alignment between text and image.