Fine-grained Controllable Video Generation via Object Appearance and Context

Understand

Text-to-video generation has shown promising results.

• However, by taking only natural languages as input, users often face difficulties in providing detailed information to precisely control the model's output.
• In this work, we propose fine-grained controllable video generation (FACTOR) to achieve detailed control.
• Specifically, FACTOR aims to control objects' appearances and context, including their location and category, in conjunction with the text prompt.