2019

DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction

Xu, Qiangeng, Wang, Weiyue, Ceylan, Duygu et al.

Understand

Reconstructing 3D shapes from single-view images has been a long-standing research problem.

In this paper, we present DISN, a Deep Implicit Surface Network which can generate a high-quality detail-rich 3D mesh from an 2D image by predicting the underlying signed distance fields.
In addition to utilizing global image features, DISN predicts the projected location for each 3D point on the 2D image, and extracts local features from the image feature maps.
Combining global and local features significantly improves the accuracy of the signed distance field prediction, especially for the detail-rich areas.

Built on

Marching cubes: A high resolution 3d surface construction algorithm
William E Lorensen and Harvey E Cline · 1987
Earlier work this paper cites.
Vega: non-linear fem deformable object simulator
Fun Shing Sin, Daniel Schroeder, and Jernej Barbič · 2013
Earlier work this paper cites.
Signed distance fields for polygon soup meshes
Hongyi Xu and Jernej Barbič · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Original
Karen Simonyan and Andrew Zisserman · 2014
Earlier work this paper cites.
Shapenet: An information-rich 3d model repository
Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu · 2015
Earlier work this paper cites.
Perspective transformer nets: Learning single-view 3d object reconstruction without 3d supervision
Xinchen Yan, Jimei Yang, Ersin Yumer, Yijie Guo, and Honglak Lee · 2016
Earlier work this paper cites.
Learning a predictable and generative vector representation for objects
R. Girdhar, D.F. Fouhey, M. Rodriguez, and A. Gupta · 2016
Earlier work this paper cites.
3d-r2n2: A unified approach for single and multi-view 3d object reconstruction
Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese · 2016
Earlier work this paper cites.
Rethinking reprojection: Closing the loop for pose-aware shape reconstruction from a single image
Rui Zhu, Hamed Kiani Galoogahi, Chaoyang Wang, and Simon Lucey · 2017
Earlier work this paper cites.
MarrNet: 3D Shape Reconstruction via 2.5D Sketches
Jiajun Wu, Yifan Wang, Tianfan Xue, Xingyuan Sun, William T Freeman, and Joshua B Tenenbaum · 2017
Earlier work this paper cites.
Multi-view supervision for single-view reconstruction via differentiable ray consistency
Shubham Tulsiani, Tinghui Zhou, Alexei A. Efros, and Jitendra Malik · 2017
Earlier work this paper cites.
Shape inpainting using 3d generative adversarial network and recurrent convolutional networks
Weiyue Wang, Qiangui Huang, Suya You, Chao Yang, and Ulrich Neumann · 2017
Earlier work this paper cites.

Similar

A point set generation network for 3d object reconstruction from a single image
Haoqiang Fan, Hao Su, and Leonidas J Guibas · 2017
Cited alongside, same era.
Shape completion using 3d-encoder-predictor cnns and shape synthesis
Angela Dai, Charles Ruizhongtai Qi, and Matthias Nießner · 2017
Cited alongside, same era.
Hierarchical surface prediction for 3d object reconstruction
Christian Häne, Shubham Tulsiani, and Jitendra Malik · 2017
Cited alongside, same era.
Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs
M. Tatarchenko, A. Dosovitskiy, and T. Brox · 2017
Cited alongside, same era.
3d-prnn: Generating shape primitives with recurrent neural networks
Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem · 2017
Cited alongside, same era.
Learning shape priors for single-view 3d completion and reconstruction
Jiajun Wu, Chengkai Zhang, Xiuming Zhang, Zhoutong Zhang, William T. Freeman, and Joshua B. Tenenbaum · 2018
Cited alongside, same era.
Learning single-view 3d reconstruction with limited pose supervision
Guandao Yang, Yin Cui, Serge Belongie, and Bharath Hariharan · 2018
Cited alongside, same era.
AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation
Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan Russell, and Mathieu Aubry · 2018
Cited alongside, same era.
Pixel2mesh: Generating 3d mesh models from single rgb images
Original
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang · 2018
Cited alongside, same era.
Learning implicit fields for generative shape modeling
Original
Zhiqin Chen and Hao Zhang · 2018
Cited alongside, same era.
Adaptive o-cnn: A patch-based deep representation of 3d shapes
Original
Peng-Shuai Wang, Chun-Yu Sun, Yang Liu, and Xin Tong · 2018
Cited alongside, same era.
Im2struct: Recovering 3d shape structure from a single rgb image
Chengjie Niu, Jun Li, and Kai Xu · 2018
Cited alongside, same era.

Then

Surfnet: Generating 3d shape surfaces using deep residual networks
Ayan Sinha, Asim Unmesh, Qixing Huang, and Karthik Ramani · 2018
Later among the works it cites.
Unsupervised learning of shape and pose with differentiable point clouds
Eldar Insafutdinov and Alexey Dosovitskiy · 2018
Later among the works it cites.
On the continuity of rotation representations in neural networks
Original
Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li · 2018
Later among the works it cites.
Neural 3d mesh renderer
Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada · 2018
Later among the works it cites.
Occupancy networks: Learning 3d reconstruction in function space
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger · 2019
Closest in time.
3dn: 3d deformation network
Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann · 2019
Closest in time.
Deepsdf: Learning continuous signed distance functions for shape representation
Original
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove · 2019
Closest in time.
A skeleton-bridged deep learning approach for generating meshes of complex topologies from single rgb images
Jiapeng Tang, Xiaoguang Han, Junyi Pan, Kui Jia, and Xin Tong · 2019
Closest in time.
Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization
Original
Shunsuke Saito, , Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li · 2019
Closest in time.
Deep rgb-d canonical correlation analysis for sparse depth completion
Yiqi Zhong, Cho-Ying Wu, Suya You, and Ulrich Neumann · 2019
Closest in time.
What do single-view 3d reconstruction networks learn?
Maxim Tatarchenko, Stephan R Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…