Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views

Understand

Automatic perception of human behaviors during social interactions is crucial for AR/VR applications, and an essential component is estimation of plausible 3D human pose and shape of our social partners from the egocentric view.

• One of the biggest challenges of this task is severe body truncation due to close social distances in egocentric scenarios, which brings large pose ambiguities for unseen body parts.
• To tackle this challenge, we propose a novel scene-conditioned diffusion method to model the body pose distribution.
• Conditioned on the 3D scene geometry, the diffusion model generates bodies in plausible human-scene interactions, with the sampling guided by a physics-based collision score to further resolve human-scene inter-penetrations.