VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation

Understand

Controllable image-to-video (I2V) generation transforms a reference image into a coherent video guided by user-specified control signals.

• While precise control over camera motion, object motion, and lighting is essential for high-fidelity creation, existing methods often treat these factors independently.
• This overlooks the physical coupling among viewpoint, geometry, and illumination in dynamic scenes, leading to visual inconsistencies such as mismatched shadows and perspective drift under simultaneous changes.
• We present VidCRAFT3, a unified and flexible I2V framework that explicitly models cross-factor interactions among geometry, motion, and illumination, enabling both independent and joint control over camera motion, object motion, and lighting direction.