Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers

Understand

In this paper, we develop novel, efficient 2D encodings for 3D geometry, which enable reconstructing full 3D shapes from a single image at high resolution.

• The key idea is to pose 3D shape reconstruction as a 2D prediction problem.
• To that end, we first develop a simple baseline network that predicts entire voxel tubes at each pixel of a reference view.
• By leveraging well-proven architectures for 2D pixel-prediction tasks, we attain state-of-the-art results, clearly outperforming purely voxel-based approaches.