MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets

Understand

Driven by powerful image diffusion models, recent research has achieved the automatic creation of 3D objects from textual or visual guidance.

• By performing score distillation sampling (SDS) iteratively across different views, these methods succeed in lifting 2D generative prior to the 3D space.
• However, such a 2D generative image prior bakes the effect of illumination and shadow into the texture.
• As a result, material maps optimized by SDS inevitably involve spurious correlated components.