Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting

Understand

3D scene generation has quickly become a challenging new research direction, fueled by consistent improvements of 2D generative diffusion models.

• Most prior work in this area generates scenes by iteratively stitching newly generated frames with existing geometry.
• These works often depend on pre-trained monocular depth estimators to lift the generated images into 3D, fusing them with the existing scene representation.
• These approaches are then often evaluated via a text metric, measuring the similarity between the generated images and a given text prompt.