Synthesizing Moving People with 3D Control

Understand

In this paper, we present a diffusion model-based framework for animating people from a single image for a given target 3D motion sequence.

• Our approach has two core components: a) learning priors about invisible parts of the human body and clothing, and b) rendering novel body poses with proper clothing and texture.
• For the first part, we learn an in-filling diffusion model to hallucinate unseen parts of a person given a single image.
• We train this model on texture map space, which makes it more sample-efficient since it is invariant to pose and viewpoint.