Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering

Understand

Differentiable rendering has paved the way to training neural networks to perform "inverse graphics" tasks such as predicting 3D geometry from monocular photographs.

• To train high performing models, most of the current approaches rely on multi-view imagery which are not readily available in practice.
• Recent Generative Adversarial Networks (GANs) that synthesize images, in contrast, seem to acquire 3D knowledge implicitly during training: object viewpoints can be manipulated by simply manipulating the latent codes.
• However, these latent codes often lack further physical interpretation and thus GANs cannot easily be inverted to perform explicit 3D reasoning.