Text Prompting for Multi-Concept Video Customization by Autoregressive Generation

Understand

We present a method for multi-concept customization of pretrained text-to-video (T2V) models.

• Intuitively, the multi-concept customized video can be derived from the (non-linear) intersection of the video manifolds of the individual concepts, which is not straightforward to find.
• We hypothesize that sequential and controlled walking towards the intersection of the video manifolds, directed by text prompting, leads to the solution.
• To do so, we generate the various concepts and their corresponding interactions, sequentially, in an autoregressive manner.