2017

SegICP: Integrated Deep Semantic Segmentation and Pose Estimation

Wong, Jay M., Kee, Vincent, Le, Tiffany et al.

Understand

Recent robotic manipulation competitions have highlighted that sophisticated robots still struggle to achieve fast and reliable perception of task-relevant objects in complex, realistic scenarios.

To improve these systems' perceptive speed and robustness, we present SegICP, a novel integrated solution to object recognition and pose estimation.
SegICP couples convolutional neural networks and multi-hypothesis point cloud registration to achieve both robust pixel-wise semantic segmentation as well as accurate and real-time 6-DOF pose estimation for relevant objects.
Our architecture achieves 1cm position error and <5^\circ$ angle error in real time without an initial seed.

Built on

P. J. Besl and N. D. McKay, “Method for registration of 3-d shapes,” in
1992
Earlier work this paper cites.
D. P. Huttenlocher, G. A. Klanderman, and W. J. Rucklidge, “Comparing images using the hausdorff distance,”
1993
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,”
1998
Earlier work this paper cites.
D. G. Lowe, “Local feature view clustering for 3d object recognition,” in
2001
Earlier work this paper cites.
S. Rusinkiewicz and M. Levoy, “Efficient variants of the icp algorithm,” in
2001
Earlier work this paper cites.
——, “Distinctive image features from scale-invariant keypoints,”
2004
Earlier work this paper cites.
S. Savarese and L. Fei-Fei, “3d generic object categorization, localization and pose estimation,” in
2007
Earlier work this paper cites.
J. Shotton, M. Johnson, and R. Cipolla, “Semantic texton forests for image categorization and segmentation,” in
2008
Earlier work this paper cites.
G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla, “Segmentation and recognition using structure from motion point clouds,” in
2008
Earlier work this paper cites.
B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “Labelme: a database and web-based tool for image annotation,”
2008
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in
Original
2009
Earlier work this paper cites.

Similar

L. Ladickỳ, P. Sturgess, K. Alahari, C. Russell, and P. H. Torr, “What, where and how many? combining object detectors and crfs,” in
2010
Cited alongside, same era.
R. B. Rusu and S. Cousins, “3D is here: Point Cloud Library (PCL),” in
2011
Cited alongside, same era.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in
2012
Cited alongside, same era.
S. Hinterstoisser, V. Lepetit, S. Ilic, S. Holzer, G. Bradski, K. Konolige, and N. Navab, “Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes,” in
2012
Cited alongside, same era.
B. Pepik, M. Stark, P. Gehler, and B. Schiele, “Teaching 3d geometry to deformable part models,” in
2012
Cited alongside, same era.
G. Pratt and J. Manzo, “The darpa robotics challenge [competitions],”
2013
Cited alongside, same era.
T. Schmidt, R. A. Newcombe, and D. Fox, “Dart: Dense articulated real-time tracking.” in
2014
Cited alongside, same era.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in
2014
Cited alongside, same era.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,”
2014
Cited alongside, same era.
J. J. Lim, A. Khosla, and A. Torralba, “Fpm: Fine pose parts-based model with 3d cad models,” in
2014
Cited alongside, same era.
P. R. Wurman and J. M. Romano, “The amazonpicking challenge 2015,”
2015
Cited alongside, same era.

Then

S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in
2015
Later among the works it cites.
V. Badrinarayanan, A. Handa, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling,”
2015
Later among the works it cites.
H. Noh, S. Hong, and B. Han, “Learning deconvolution network for semantic segmentation,” in
2015
Later among the works it cites.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in
2015
Later among the works it cites.
F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,”
2015
Later among the works it cites.
A. Zeng, K.-T. Yu, S. Song, D. Suo, E. Walker Jr, A. Rodriguez, and J. Xiao, “Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge,”
2016
Later among the works it cites.
Q.-Y. Zhou, J. Park, and V. Koltun, “Fast global registration,” in
2016
Later among the works it cites.
M.-Y. Liu, O. Tuzel, A. Veeraraghavan, Y. Taguchi, T. K. Marks, and R. Chellappa, “Fast object localization and pose estimation in heavy clutter for robotic bin picking,”
2016
Later among the works it cites.
B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Semantic understanding of scenes through the ade20k dataset,”
Original
2016
Later among the works it cites.
C. Yuan, X. Yu, and Z. Luo, “3d point cloud matching based on principal component analysis and iterative closest point algorithm,” in
2016
Later among the works it cites.
R. Rios-Cabrera and T. Tuytelaars, “Discriminatively trained templates for 3d object detection: A real time scalable approach,” in
2055
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…