Integrating Meshes and 3D Gaussians for Indoor Scene Reconstruction with SAM Mask Guidance

Understand

We present a novel approach for 3D indoor scene reconstruction that combines 3D Gaussian Splatting (3DGS) with mesh representations.

• We use meshes for the room layout of the indoor scene, such as walls, ceilings, and floors, while employing 3D Gaussians for other objects.
• This hybrid approach leverages the strengths of both representations, offering enhanced flexibility and ease of editing.
• However, joint training of meshes and 3D Gaussians is challenging because it is not clear which primitive should affect which part of the rendered image.