MaterialGAN: Reflectance Capture using a Generative SVBRDF Model

Understand

We address the problem of reconstructing spatially-varying BRDFs from a small set of image measurements.

• This is a fundamentally under-constrained problem, and previous work has relied on using various regularization priors or on capturing many images to produce plausible results.
• In this work, we present MaterialGAN, a deep generative convolutional network based on StyleGAN2, trained to synthesize realistic SVBRDF parameter maps.
• We show that MaterialGAN can be used as a powerful material prior in an inverse rendering framework: we optimize in its latent representation to generate material maps that match the appearance of the captured images when rendered.