2019

LatticeNet: Fast Point Cloud Segmentation Using Permutohedral Lattices

Rosu, Radu Alexandru, Schütt, Peer, Quenzel, Jan et al.

Understand

Deep convolutional neural networks (CNNs) have shown outstanding performance in the task of semantically segmenting images.

However, applying the same methods on 3D data still poses challenges due to the heavy memory requirements and the lack of structured data.
Here, we propose LatticeNet, a novel approach for 3D semantic segmentation, which takes as input raw point clouds.
A PointNet describes the local geometry which we embed into a sparse permutohedral lattice.

Built on

Some useful properties of the permutohedral lattice for Gaussian filtering
Jongmin Baek and Andrew Adams · 2009
Earlier work this paper cites.
Real-time 3D reconstruction at scale using voxel hashing
Matthias Nießner, Michael Zollhöfer, Shahram Izadi, and Marc Stamminger · 2013
Earlier work this paper cites.
cuDNN: Efficient primitives for deep learning
Original
Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer · 2014
Earlier work this paper cites.
Fast bilateral-space stereo for synthetic defocus — Supplemental material
Jonathan T Barron, Andrew Adams, S YiChang, and Carlos Hernández · 2015
Earlier work this paper cites.
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, and Trevor Darrell · 2015
Earlier work this paper cites.
Geodesic convolutional neural networks on Riemannian manifolds
Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst · 2015
Earlier work this paper cites.
U-Net: Convolutional networks for biomedical image segmentation
Original
Olaf Ronneberger, Philipp Fischer, and Thomas Brox · 2015
Earlier work this paper cites.
Convolutional neural networks on graphs with fast localized spectral filtering
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst · 2016
Earlier work this paper cites.
Deep Learning with Sets and Point Clouds
Original
Siamak Ravanbakhsh, Jeff G. Schneider, and Barnabás Póczos · 2016
Earlier work this paper cites.
A scalable active framework for region annotation in 3D shape collections
Li Yi, Vladimir G Kim, Duygu Ceylan, I Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, Leonidas Guibas, et al · 2016
Earlier work this paper cites.
Rethinking atrous convolution for semantic image segmentation
Original
Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam · 2017
Earlier work this paper cites.
ScanNet: Richly-annotated 3D reconstructions of indoor scenes
Original
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner · 2017
Earlier work this paper cites.

Similar

Densely connected convolutional networks
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger · 2017
Cited alongside, same era.
RefineNet: Multi-path refinement networks for high-resolution semantic segmentation
Guosheng Lin, Anton Milan, Chunhua Shen, and Ian Reid · 2017
Cited alongside, same era.
Geometric deep learning on graphs and manifolds using mixture model CNNs
Original
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein · 2017
Cited alongside, same era.
Automatic Differentiation in PyTorch
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer · 2017
Cited alongside, same era.
SEGCloud: Semantic segmentation of 3D point clouds
Original
Lyne Tchapmi, Christopher Choy, Iro Armeni, JunYoung Gwak, and Silvio Savarese · 2017
Cited alongside, same era.
Deep sets
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola · 2017
Cited alongside, same era.
Deep RBFNet: Point cloud feature learning using radial basis functions
Original
Weikai Chen, Xiaoguang Han, Guanbin Li, Chao Chen, Jun Xing, Yajie Zhao, and Hao Li · 2018
Cited alongside, same era.
3DMV : Joint 3D-multi-view prediction for 3D semantic scene segmentation
Original
Angela Dai and Matthias Nießner · 2018
Cited alongside, same era.
3D semantic segmentation with submanifold sparse convolutional networks
Benjamin Graham, Martin Engelcke, and Laurens van der Maaten · 2018
Cited alongside, same era.
PointCNN: Convolution on x-transformed points
Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen · 2018
Cited alongside, same era.
Fully-convolutional point networks for large-scale point clouds
Original
Dario Rethage, Johanna Wald, Jurgen Sturm, Nassir Navab, and Federico Tombari · 2018
Cited alongside, same era.
SplatNet: Sparse lattice networks for point cloud processing
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, and Jan Kautz · 2018
Cited alongside, same era.

Then

Tangent convolutions for dense prediction in 3D
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, and Qian-Yi Zhou · 2018
Later among the works it cites.
Deep parametric continuous convolutional neural networks
Shenlong Wang, Simon Suo, Wei-Chiu Ma, Andrei Pokrovsky, and Raquel Urtasun · 2018
Later among the works it cites.
SqueezeSegv2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a lidar point cloud
Original
Bichen Wu, Xuanyu Zhou, Sicheng Zhao, Xiangyu Yue, and Kurt Keutzer · 2018
Later among the works it cites.
Group normalization
Yuxin Wu and Kaiming He · 2018
Later among the works it cites.
SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences
Original
Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Juergen Gall · 2019
Closest in time.
4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks
Original
Christopher Choy, JunYoung Gwak, and Silvio Savarese · 2019
Closest in time.
HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds
Original
Xiuye Gu, Yijie Wang, Chongruo Wu, Yong Jae Lee, and Panqu Wang · 2019
Closest in time.
TextureNet: Consistent local parametrizations for learning from high-resolution signals on meshes
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, and Leonidas J Guibas · 2019
Closest in time.
Real-time progressive 3D semantic segmentation for indoor scenes
Original
Quang-Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung · 2019
Closest in time.
Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence
Original
Patrick Stotko, Stefan Krumpen, Michael Weinmann, and Reinhard Klein · 2019
Closest in time.
Bonn Activity Maps: Dataset Description
Original
Julian Tanke, Oh-Hun Kwon, Patrick Stotko, Radu Alexandru Rosu, Michael Weinmann, Hassan Errami, Sven Behnke, Maren Bennewitz, Reinhard Klein, Andreas Weber, et al · 2019
Closest in time.
PointConv: Deep convolutional networks on 3D point clouds
Wenxuan Wu, Zhongang Qi, and Li Fuxin · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…