2019

6-DOF GraspNet: Variational Grasp Generation for Object Manipulation

Mousavian, Arsalan, Eppner, Clemens, Fox, Dieter

Understand

Generating grasp poses is a crucial component for any robot object manipulation task.

In this work, we formulate the problem of grasp generation as sampling a set of grasps using a variational autoencoder and assess and refine the sampled grasps using a grasp evaluator model.
Both Grasp Sampler and Grasp Refinement networks take 3D point clouds observed by a depth camera as input.
We evaluate our approach in simulation and real-world robot experiments.

Built on

Learning grasp strategies with partial shape information
Ashutosh Saxena, Lawson LS Wong, and Andrew Y Ng · 2008
Earlier work this paper cites.
Learning grasping points with shape context
Jeannette Bohg and Danica Kragic · 2010
Earlier work this paper cites.
3d is here: Point cloud library (pcl)
R. B. Rusu and S. Cousins · 2011
Earlier work this paper cites.
Template-based learning of grasp selection
Alexander Herzog, Peter Pastor, Mrinal Kalakrishnan, Ludovic Righetti, Tamim Asfour, and Stefan Schaal · 2012
Earlier work this paper cites.
Data-driven grasp synthesis—a survey
Jeannette Bohg, Antonio Morales, Tamim Asfour, and Danica Kragic · 2014
Earlier work this paper cites.
Generative adversarial networks
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio · 2014
Earlier work this paper cites.
Learning rich features from RGB-D images for object detection and segmentation
Saurabh Gupta, Ross Girshick, Pablo Arbelaez, and Jitendra Malik · 2014
Earlier work this paper cites.
Auto-encoding variational bayes
Diederik P. Kingma and Max Welling · 2014
Earlier work this paper cites.
Unified particle physics for real-time applications
Miles Macklin, Matthias Müller, Nuttapong Chentanez, and Tae-Yong Kim · 2014
Earlier work this paper cites.
Shapenet: An information-rich 3d model repository
Original
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al · 2015
Earlier work this paper cites.
Deep convolutional inverse graphics network
Tejas D Kulkarni, William F. Whitney, Pushmeet Kohli, and Josh Tenenbaum · 2015
Earlier work this paper cites.
Deep learning for detecting robotic grasps
Ian Lenz, Honglak Lee, and Ashutosh Saxena · 2015
Earlier work this paper cites.

Similar

Voxnet: A 3d convolutional neural network for real-time object recognition
Daniel Maturana and Sebastian Scherer · 2015
Cited alongside, same era.
Real-time grasp detection using convolutional neural networks
Joseph Redmon and Anelia Angelova · 2015
Cited alongside, same era.
Learning structured output representation using deep conditional generative models
Kihyuk Sohn, Honglak Lee, and Xinchen Yan · 2015
Cited alongside, same era.
Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours
L. Pinto and A. Gupta · 2016
Cited alongside, same era.
Pointnet: Deep learning on point sets for 3d classification and segmentation
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas · 2016
Cited alongside, same era.
An uncertain future: Forecasting from static images using variational autoencoders
Jacob Walker, Carl Doersch, Abhinav Gupta, and Martial Hebert · 2016
Cited alongside, same era.
Desire: Distant future prediction in dynamic scenes with interacting agents
Namhoon Lee, Wongun Choi, Paul Vernaza, Christopher B. Choy, Philip H. S. Torr, and Manmohan Chandraker · 2017
Cited alongside, same era.
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection
Sergey Levine, Peter Pastor Sampedro, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen · 2017
Cited alongside, same era.
Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics
Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, and Ken Goldberg · 2017
Cited alongside, same era.
Pointnet++: Deep hierarchical feature learning on point sets in a metric space
Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas · 2017
Cited alongside, same era.
Grasp pose detection in point clouds
Andreas ten Pas, Marcus Gualtieri, Kate Saenko, and Robert Platt · 2017
Cited alongside, same era.
Learning 6-dof grasping interaction via deep geometry-aware 3d representations
Original
Xinchen Yan, Jasmine Hsu, Mohi Khansari, Yunfei Bai, Arkanath Pathak, Abhinav Gupta, James Davidson, and Honglak Lee · 2017
Cited alongside, same era.

Then

6dof grasp planning by optimizing a deep learning scoring function
Yilun Zhou and Kris Hauser · 2017
Later among the works it cites.
Learning object grasping for soft robot hands
Changhyun Choi, Wilko Schwarting, Joseph DelPreto, and Daniela Rus · 2018
Later among the works it cites.
Scalable deep reinforcement learning for vision-based robotic manipulation
Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, and Sergey Levine · 2018
Later among the works it cites.
Planning multi-fingered grasps as probabilistic inference in a learned deep network
Original
Qingkai Lu, Kautilya Chenna, Balakumar Sundaralingam, and Tucker Hermans · 2018
Later among the works it cites.
Frustum pointnets for 3d object detection from rgb-d data
Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas · 2018
Later among the works it cites.
Grasping of unknown objects using deep convolutional neural networks based on depth images
Philipp Schmidt, Nikolaus Vahrenkamp, Mirko Wächter, and Tamim Asfour · 2018
Later among the works it cites.
SPLATNet: Sparse lattice networks for point cloud processing
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, and Jan Kautz · 2018
Later among the works it cites.
Dynamic graph cnn for learning on point clouds
Yu Wang, Sun Yongbin, Ziwei Liu, Sanjay Sarma, Micheal Bronstein, and Justin Solomon · 2018
Later among the works it cites.
Pointfusion: Deep sensor fusion for 3d bounding box estimation
Danfei Xu, Ashesh Jain, and Anguelov. Dragomir · 2018
Later among the works it cites.
Learning synergies between pushing and grasping with self-supervised deep reinforcement learning
Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser · 2018
Later among the works it cites.
Pointnetgpd: Detecting grasp configurations from point sets
Hongzhuo Liang, Xiaojian Ma, Shuang Li, Michael Görner, Song Tang, Bin Fang, Fuchun Sun, and Jianwei Zhang · 2019
Closest in time.
Generating Grasp Poses for a High-DOF Gripper Using Neural Networks
Original
Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, and Dinesh Manocha · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…