Zero3D: Semantic-Driven Multi-Category 3D Shape Generation

Understand

Semantic-driven 3D shape generation aims to generate 3D objects conditioned on text.

• Previous works face problems with single-category generation, low-frequency 3D details, and requiring a large number of paired datasets for training.
• To tackle these challenges, we propose a multi-category conditional diffusion model.
• Specifically, 1) to alleviate the problem of lack of large-scale paired data, we bridge the text, 2D image and 3D shape based on the pre-trained CLIP model, and 2) to obtain the multi-category 3D shape feature, we apply the conditional flow model to generate 3D shape vector conditioned on CLIP embedding.