MaterialFusion: Enhancing Inverse Rendering with Material Diffusion Priors

Understand

Recent works in inverse rendering have shown promise in using multi-view images of an object to recover shape, albedo, and materials.

• However, the recovered components often fail to render accurately under new lighting conditions due to the intrinsic challenge of disentangling albedo and material properties from input images.
• To address this challenge, we introduce MaterialFusion, an enhanced conventional 3D inverse rendering pipeline that incorporates a 2D prior on texture and material properties.
• We present StableMaterial, a 2D diffusion model prior that refines multi-lit data to estimate the most likely albedo and material from given input appearances.