Neural Shape Compiler: A Unified Framework for Transforming between Text, Point Cloud, and Program

Understand

3D shapes have complementary abstractions from low-level geometry to part-based hierarchies to languages, which convey different levels of information.

• This paper presents a unified framework to translate between pairs of shape abstractions: $\textit{Text}$ $\Longleftrightarrow$ $\textit{Point Cloud}$ $\Longleftrightarrow$ $\textit{Program}$.
• We propose $\textbf{Neural Shape Compiler}$ to model the abstraction transformation as a conditional generation process.
• It converts 3D shapes of three abstract types into unified discrete shape code, transforms each shape code into code of other abstract types through the proposed $\textit{ShapeCode Transformer}$, and decodes them to output the target shape abstraction.