Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction

Understand

We propose a novel method to reconstruct the 3D shapes of transparent objects using hand-held captured images under natural light conditions.

• It combines the advantage of explicit mesh and multi-layer perceptron (MLP) network, a hybrid representation, to simplify the capture setting used in recent contributions.
• After obtaining an initial shape through the multi-view silhouettes, we introduce surface-based local MLPs to encode the vertex displacement field (VDF) for the reconstruction of surface details.
• The design of local MLPs allows to represent the VDF in a piece-wise manner using two layer MLP networks, which is beneficial to the optimization algorithm.