BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images

Understand

We present BlockGAN, an image generative model that learns object-aware 3D scene representations directly from unlabelled 2D images.

• Current work on scene representation learning either ignores scene background or treats the whole scene as one object.
• Meanwhile, work that considers scene compositionality treats scene objects only as image patches or 2D layers with alpha maps.
• Inspired by the computer graphics pipeline, we design BlockGAN to learn to first generate 3D features of background and foreground objects, then combine them into 3D features for the wholes cene, and finally render them into realistic images.