LVCD: Reference-based Lineart Video Colorization with Diffusion Models

Understand

We propose the first video diffusion framework for reference-based lineart video colorization.

• Unlike previous works that rely solely on image generative models to colorize lineart frame by frame, our approach leverages a large-scale pretrained video diffusion model to generate colorized animation videos.
• This approach leads to more temporally consistent results and is better equipped to handle large motions.
• Firstly, we introduce Sketch-guided ControlNet which provides additional control to finetune an image-to-video diffusion model for controllable video synthesis, enabling the generation of animation videos conditioned on lineart.