2018

Learning Implicit Fields for Generative Shape Modeling

Chen, Zhiqin, Zhang, Hao

Understand

We advocate the use of implicit fields for learning generative models of shapes and introduce an implicit field decoder, called IM-NET, for shape generation, aimed at improving the visual quality of the generated shapes.

An implicit field assigns a value to each point in 3D space, so that a shape can be extracted as an iso-surface.
IM-NET is trained to perform this assignment by means of a binary classifier.
Specifically, it takes a point coordinate, along with a feature vector encoding a shape, and outputs a value which indicates whether the point is outside the shape or not.

Built on

Marching cubes: A high resolution 3d surface construction algorithm
W. E. Lorensen and H. E. Cline · 1987
Earlier work this paper cites.
Approximation capabilities of multilayer feedforward networks
K. Hornik · 1991
Earlier work this paper cites.
The ball-pivoting algorithm for surface reconstruction
F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin · 1999
Earlier work this paper cites.
On visual similarity based 3d model retrieval
D.-Y. Chen, X.-P. Tian, Y.-T. Shen, and M. Ouhyoung · 2003
Earlier work this paper cites.
The graph neural network model
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini · 2009
Earlier work this paper cites.
Efficient and flexible sampling with blue noise properties of triangular meshes
M. Corsini, P. Cignoni, and R. Scopigno · 2012
Earlier work this paper cites.
Better mixing via deep representations
Y. Bengio, G. Mesnil, Y. Dauphin, and S. Rifai · 2013
Earlier work this paper cites.
Deep generative stochastic networks trainable by backprop
Y. Bengio, E. Laufer, G. Alain, and J. Yosinski · 2014
Earlier work this paper cites.
Learning a deep convolutional network for image super-resolution
C. Dong, C. C. Loy, K. He, and X. Tang · 2014
Earlier work this paper cites.
Generative adversarial nets
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio · 2014
Earlier work this paper cites.
Auto-encoding variational bayes
D. P. Kingma and M. Welling · 2014
Earlier work this paper cites.
ShapeNet: An Information-Rich 3D Model Repository
Original
A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu · 2015
Earlier work this paper cites.
High-pass quantization for mesh encoding
D. C.-O. Olga Sorkine and S. Toledo · 2015
Earlier work this paper cites.
Multi-view convolutional neural networks for 3d shape recognition
H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller · 2015
Earlier work this paper cites.
3d-r2n2: A unified approach for single and multi-view 3d object reconstruction
C. B. Choy, D. Xu, J. Gwak, K. Chen, and S. Savarese · 2016
Earlier work this paper cites.
Learning a predictable and generative vector representation for objects
R. Girdhar, D. F. Fouhey, M. Rodriguez, and A. Gupta · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Unsupervised representation learning with deep convolutional generative adversarial networks
A. Radford, L. Metz, and S. Chintala · 2016
Earlier work this paper cites.

Similar

Deep learning 3d shape surfaces using geometry images
A. Sinha, J. Bai, and K. Ramani · 2016
Cited alongside, same era.
Conditional image generation with pixelcnn decoders
A. van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves, et al · 2016
Cited alongside, same era.
Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling
J. Wu, C. Zhang, T. Xue, B. Freeman, and J. Tenenbaum · 2016
Cited alongside, same era.
Towards principled methods for training generative adversarial networks
M. Arjovsky and L. Bottou · 2017
Cited alongside, same era.
Wasserstein generative adversarial networks
M. Arjovsky, S. Chintala, and L. Bottou · 2017
Cited alongside, same era.
Synthesizing 3d shapes via modeling multi-view depth maps and silhouettes with deep generative networks
A. Arsalan Soltani, H. Huang, J. Wu, T. D. Kulkarni, and J. B. Tenenbaum · 2017
Cited alongside, same era.
A point set generation network for 3d object reconstruction from a single image
H. Fan, H. Su, and L. J. Guibas · 2017
Cited alongside, same era.
Improved training of wasserstein gans
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville · 2017
Cited alongside, same era.
Hierarchical surface prediction for 3d object reconstruction
C. Häne, S. Tulsiani, and J. Malik · 2017
Cited alongside, same era.
Photo-realistic single image super-resolution using a generative adversarial network
C. Ledig, L. Theis, F. Huszar, J. Caballero, A. P. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi · 2017
Cited alongside, same era.
Alice: Towards understanding adversarial learning for joint distribution matching
C. Li, H. Liu, C. Chen, Y. Pu, L. Chen, R. Henao, and L. Carin · 2017
Cited alongside, same era.
Grass: Generative recursive autoencoders for shape structures
J. Li, K. Xu, S. Chaudhuri, E. Yumer, H. Zhang, and L. Guibas · 2017
Cited alongside, same era.

Then

O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis
P.-S. Wang, Y. Liu, Y.-X. Guo, C.-Y. Sun, and X. Tong · 2017
Later among the works it cites.
Learning representations and generative models for 3d point clouds
P. Achlioptas, O. Diamanti, I. Mitliagkas, and L. J. Guibas · 2018
Closest in time.
Multiresolution tree networks for 3d point cloud processing
M. Gadelha, R. Wang, and S. Maji · 2018
Closest in time.
Atlasnet: A papier-mâché approach to learning 3d surface generation
T. Groueix, M. Fisher, V. G. Kim, B. Russell, and M. Aubry · 2018
Closest in time.
Deep volumetric video from very sparse multi-view performance capture
Z. Huang, T. Li, W. Chen, Y. Zhao, J. Xing, C. LeGendre, L. Luo, C. Ma, and H. Li · 2018
Closest in time.
Progressive growing of gans for improved quality, stability, and variation
T. Karras, T. Aila, S. Laine, and J. Lehtinen · 2018
Closest in time.
Glow: Generative flow with invertible 1x1 convolutions
Original
D. P. Kingma and P. Dhariwal · 2018
Closest in time.
Deep marching cubes: Learning explicit surface representations
Y. Liao, S. Donné, and A. Geiger · 2018
Closest in time.
Learning efficient point cloud generation for dense 3d object reconstruction
C.-H. Lin, C. Kong, and S. Lucey · 2018
Closest in time.
Pixel2mesh: Generating 3d mesh models from single rgb images
N. Wang, Y. Zhang, Z. Li, Y. Fu, W. Liu, and Y.-G. Jiang · 2018
Closest in time.
Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes
P.-S. Wang, C.-Y. Sun, Y. Liu, and X. Tong · 2018
Closest in time.
Learning shape priors for single-view 3d completion and reconstruction
J. Wu, C. Zhang, X. Zhang, Z. Zhang, W. T. Freeman, and J. B. Tenenbaum · 2018
Closest in time.
Foldingnet: Point cloud auto-encoder via deep grid deformation
Y. Yang, C. Feng, Y. Shen, and D. Tian · 2018
Closest in time.
P2P-NET: Bidirectional point displacement net for shape transform
K. Yin, H. Huang, D. Cohen-Or, and H. Zhang · 2018
Closest in time.
SCORES: Shape composition with recursive substructure priors
C. Zhu, K. Xu, S. Chaudhuri, R. Yi, and H. Zhang · 2018
Closest in time.
Deepsdf: Learning continuous signed distance functions for shape representation
J. Joon Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove · 2019
Closest in time.
Occupancy networks: Learning 3d reconstruction in function space
L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…