2019

Deformable Filter Convolution for Point Cloud Reasoning

Xiong, Yuwen, Ren, Mengye, Liao, Renjie et al.

Understand

Point clouds are the native output of many real-world 3D sensors.

To borrow the success of 2D convolutional network architectures, a majority of popular 3D perception models voxelize the points, which can result in a loss of local geometric details that cannot be recovered.
In this paper, we propose a novel learnable convolution layer for processing 3D point cloud data directly.
Instead of discretizing points into fixed voxels, we deform our learnable 3D filters to match with the point cloud shape.

Built on

Distinctive image features from scale-invariant keypoints
D. G. Lowe · 2004
Earlier work this paper cites.
Are we ready for autonomous driving? the KITTI vision benchmark suite
A. Geiger, P. Lenz, and R. Urtasun · 2012
Earlier work this paper cites.
Spectral networks and locally connected networks on graphs
J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun · 2014
Earlier work this paper cites.
Visualizing and understanding convolutional networks
M. D. Zeiler and R. Fergus · 2014
Earlier work this paper cites.
Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks
D. Boscaini, J. Masci, S. Melzi, M. M. Bronstein, U. Castellani, and P. Vandergheynst · 2015
Earlier work this paper cites.
Voxnet: A 3d convolutional neural network for real-time object recognition
D. Maturana and S. Scherer · 2015
Earlier work this paper cites.
U-net: Convolutional networks for biomedical image segmentation
O. Ronneberger, P. Fischer, and T. Brox · 2015
Earlier work this paper cites.
Multi-view convolutional neural networks for 3d shape recognition
H. Su, S. Maji, E. Kalogerakis, and E. G. Learned-Miller · 2015
Earlier work this paper cites.
3d shapenets: A deep representation for volumetric shapes
Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao · 2015
Earlier work this paper cites.
Vehicle detection from 3d lidar using fully convolutional network
B. Li, T. Zhang, and T. Xia · 2016
Earlier work this paper cites.
Gated graph sequence neural networks
Y. Li, D. Tarlow, M. Brockschmidt, and R. S. Zemel · 2016
Earlier work this paper cites.
Volumetric and multi-view cnns for object classification on 3d data
C. R. Qi, H. Su, M. Nießner, A. Dai, M. Yan, and L. J. Guibas · 2016
Earlier work this paper cites.
Multi-view 3d object detection network for autonomous driving
X. Chen, H. Ma, J. Wan, B. Li, and T. Xia · 2017
Earlier work this paper cites.
Xception: Deep learning with depthwise separable convolutions
F. Chollet · 2017
Earlier work this paper cites.
Deformable convolutional networks
J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei · 2017
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
T. N. Kipf and M. Welling · 2017
Earlier work this paper cites.

Similar

Escape from cells: Deep kd-networks for the recognition of 3d point cloud models
R. Klokov and V. S. Lempitsky · 2017
Cited alongside, same era.
Pointnet: Deep learning on point sets for 3d classification and segmentation
C. R. Qi, H. Su, K. Mo, and L. J. Guibas · 2017
Cited alongside, same era.
Pointnet++: Deep hierarchical feature learning on point sets in a metric space
C. R. Qi, L. Yi, H. Su, and L. J. Guibas · 2017
Cited alongside, same era.
3d graph neural networks for rgbd semantic segmentation
X. Qi, R. Liao, J. Jia, S. Fidler, and R. Urtasun · 2017
Cited alongside, same era.
Octnet: Learning deep 3d representations at high resolutions
G. Riegler, A. O. Ulusoy, and A. Geiger · 2017
Cited alongside, same era.
Schnet: A continuous-filter convolutional neural network for modeling quantum interactions
K. Schütt, P. Kindermans, H. E. S. Felix, S. Chmiela, A. Tkatchenko, and K. Müller · 2017
Cited alongside, same era.
Dynamic edge-conditioned filters in convolutional neural networks on graphs
M. Simonovsky and N. Komodakis · 2017
Cited alongside, same era.
O-CNN: octree-based convolutional neural networks for 3d shape analysis
P. Wang, Y. Liu, Y. Guo, C. Sun, and X. Tong · 2017
Cited alongside, same era.
Deep sets
M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Póczos, R. R. Salakhutdinov, and A. J. Smola · 2017
Cited alongside, same era.
Point convolutional neural networks by extension operators
M. Atzmon, H. Maron, and Y. Lipman · 2018
Cited alongside, same era.
Flex-convolution (million-scale point-cloud learning beyond grid-worlds)
F. Groh, P. Wieschollek, and H. P. A. Lensch · 2018
Cited alongside, same era.
Monte carlo convolution for learning on non-uniformly sampled point clouds
P. Hermosilla, T. Ritschel, P. Vázquez, A. Vinacua, and T. Ropinski · 2018
Cited alongside, same era.

Then

Pointcnn: Convolution on x-transformed points
Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen · 2018
Later among the works it cites.
Pointcnn: Convolution on x-transformed points
Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen · 2018
Later among the works it cites.
Deep continuous fusion for multi-sensor 3d object detection
M. Liang, B. Yang, S. Wang, and R. Urtasun · 2018
Later among the works it cites.
Fast and furious: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net
W. Luo, B. Yang, and R. Urtasun · 2018
Later among the works it cites.
Frustum pointnets for 3d object detection from RGB-D data
C. R. Qi, W. Liu, C. Wu, H. Su, and L. J. Guibas · 2018
Later among the works it cites.
Sbnet: Sparse blocks network for fast inference
M. Ren, A. Pokrovsky, B. Yang, and R. Urtasun · 2018
Later among the works it cites.
Mining point cloud local structures by kernel correlation and graph pooling
Y. Shen, C. Feng, Y. Yang, and D. Tian · 2018
Later among the works it cites.
Deep parametric continuous convolutional neural networks
S. Wang, S. Suo, W. Ma, A. Pokrovsky, and R. Urtasun · 2018
Later among the works it cites.
Dynamic graph CNN for learning on point clouds
Original
Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon · 2018
Later among the works it cites.
Spidercnn: Deep learning on point sets with parameterized convolutional filters
Y. Xu, T. Fan, M. Xu, L. Zeng, and Y. Qiao · 2018
Later among the works it cites.
HDNET: exploiting HD maps for 3d object detection
B. Yang, M. Liang, and R. Urtasun · 2018
Later among the works it cites.
PIXOR: real-time 3d object detection from point clouds
B. Yang, W. Luo, and R. Urtasun · 2018
Later among the works it cites.
Efficient convolutions for real-time semantic segmentation of 3d point clouds
Original
C. Zhang, W. Luo, and R. Urtasun · 2018
Later among the works it cites.
Kpconv: Flexible and deformable convolution for point clouds
Original
H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas · 2019
Closest in time.
Dynamic graph cnn for learning on point clouds
Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…