CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibility

Understand

Recent advancements in video generation have been remarkable, yet many existing methods struggle with issues of consistency and poor text-video alignment.

• Moreover, the field lacks effective techniques for text-guided video inpainting, a stark contrast to the well-explored domain of text-guided image inpainting.
• To this end, this paper proposes a novel text-guided video inpainting model that achieves better consistency, controllability and compatibility.
• Specifically, we introduce a simple but efficient motion capture module to preserve motion consistency, and design an instance-aware region selection instead of a random region selection to obtain better textual controllability, and utilize a novel strategy to inject some personalized models into our CoCoCo model and thus obtain better model compatibility.