UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding

Understand

Despite the impressive progress on understanding and generating images shown by the recent unified architectures, the integration of 3D tasks remains challenging and largely unexplored.

• In this paper, we introduce UniUGG, the first unified understanding and generation framework for 3D modalities.
• Our unified framework employs an LLM to comprehend and decode sentences and 3D representations.
• At its core, we propose a spatial decoder leveraging a latent diffusion model to generate high-quality 3D representations.