Blocks2World: Controlling Realistic Scenes with Editable Primitives

Understand

We present Blocks2World, a novel method for 3D scene rendering and editing that leverages a two-step process: convex decomposition of images and conditioned synthesis.

• Our technique begins by extracting 3D parallelepipeds from various objects in a given scene using convex decomposition, thus obtaining a primitive representation of the scene.
• These primitives are then utilized to generate paired data through simple ray-traced depth maps.
• The next stage involves training a conditioned model that learns to generate images from the 2D-rendered convex primitives.