Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamics

Understand

We introduce Puppet-Master, an interactive video generator that captures the internal, part-level motion of objects, serving as a proxy for modeling object dynamics universally.

• Given an image of an object and a set of "drags" specifying the trajectory of a few points on the object, the model synthesizes a video where the object's parts move accordingly.
• To build Puppet-Master, we extend a pre-trained image-to-video generator to encode the input drags.
• We also propose all-to-first attention, an alternative to conventional spatial attention that mitigates artifacts caused by fine-tuning a video generator on out-of-domain data.