2020

A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects

Simeonov, Anthony, Du, Yilun, Kim, Beomjoon et al.

Understand

We present a framework for solving long-horizon planning problems involving manipulation of rigid objects that operates directly from a point-cloud observation, i.e.

without prior object models.
Our method plans in the space of object subgoals and frees the planner from reasoning about robot-object interaction dynamics by relying on a set of generalizable manipulation primitives.
We show that for rigid bodies, this abstraction can be realized using low-level manipulation skills that maintain sticking contact with the object and represent subgoals as 3D transformations.

Built on

Besl and neil mckay, a methode for registration of 3d shapes
J. Paul · 1992
Earlier work this paper cites.
Rapidly-exploring random trees: A new tool for path planning
S. M. LaValle · 1998
Earlier work this paper cites.
Rrt-connect: An efficient approach to single-query path planning
J. J. Kuffner and S. M. LaValle · 2000
Earlier work this paper cites.
Effective sampling and distance metrics for 3d rigid body path planning
J. J. Kuffner · 2004
Earlier work this paper cites.
A hybrid approach to intricate motion, manipulation, and task planning
S. Cambon, R. Alami, and F. Gravot · 2009
Earlier work this paper cites.
Metrics for 3d rotations: Comparison and analysis
D. Q. Huynh · 2009
Earlier work this paper cites.
Hierarchical task and motion planning in the now
L. P. Kaelbling and T. Lozano-Pérez · 2011
Earlier work this paper cites.
Affordance prediction via learned object attributes
T. Hermans, J. M. Rehg, and A. Bobick · 2011
Earlier work this paper cites.
3D is here: Point Cloud Library (PCL)
R. B. Rusu and S. Cousins · 2011
Earlier work this paper cites.
Auto-encoding variational bayes
Original
D. P. Kingma and M. Welling · 2013
Earlier work this paper cites.
Ffrob: Leveraging symbolic planning for efficient task and motion planning
C. R. Garrett, T. Lozano-Peréz, and L. P. Kaelbling · 2014
Earlier work this paper cites.
A constraint-based method for solving sequential manipulation planning problems
T. Lozano-Pérez and L. P. Kaelbling · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
D. P. Kingma and J. Ba · 2014
Earlier work this paper cites.
Logic-geometric programming: An optimization-based approach to combined task and motion planning
M. Toussaint · 2015
Earlier work this paper cites.
Learning structured output representation using deep conditional generative models
K. Sohn, H. Lee, and X. Yan · 2015
Earlier work this paper cites.
Learning to poke by poking: Experiential learning of intuitive physics
P. Agrawal, A. V. Nair, P. Abbeel, J. Malik, and S. Levine · 2016
Earlier work this paper cites.
Pybullet, a python module for physics simulation for games, robotics and machine learning
E. Coumans and Y. Bai · 2016
Earlier work this paper cites.

Similar

Guided search for task and motion plans using learned heuristics
R. Chitnis, D. Hadfield-Menell, A. Gupta, S. Srivastava, E. Groshev, C. Lin, and P. Abbeel · 2016
Cited alongside, same era.
Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics
Original
J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea, and K. Goldberg · 2017
Cited alongside, same era.
Deep visual foresight for planning robot motion
C. Finn and S. Levine · 2017
Cited alongside, same era.
Sample-based methods for factored task and motion planning
C. R. Garrett, T. Lozano-Perez, and L. P. Kaelbling · 2017
Cited alongside, same era.
Pointnet++: Deep hierarchical feature learning on point sets in a metric space
C. R. Qi, L. Yi, H. Su, and L. J. Guibas · 2017
Cited alongside, same era.
Graph attention networks
Original
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio · 2017
Cited alongside, same era.
Mask r-cnn
K. He, G. Gkioxari, P. Dollár, and R. Girshick · 2017
Cited alongside, same era.
Se3-nets: Learning rigid body motion using deep neural networks
A. Byravan and D. Fox · 2017
Cited alongside, same era.
Learning to guide task and motion planning using score-space representation
B. Kim, L. P. Kaelbling, and T. Lozano-Pérez · 2017
Cited alongside, same era.
Automatic differentiation in pytorch
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer · 2017
Cited alongside, same era.
Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching
A. Zeng, S. Song, K.-T. Yu, E. Donlon, F. R. Hogan, M. Bauza, D. Ma, O. Taylor, M. Liu, E. Romo, et al · 2018
Cited alongside, same era.
Visual foresight: Model-based deep reinforcement learning for vision-based robotic control
Original
F. Ebert, C. Finn, S. Dasari, A. Xie, A. Lee, and S. Levine · 2018
Cited alongside, same era.

Then

Accurate vision-based manipulation through contact reasoning
Original
A. Kloss, M. Bauza, J. Wu, J. B. Tenenbaum, A. Rodriguez, and J. Bohg · 2019
Later among the works it cites.
Learning value functions with relational state representations for guiding task-and-motion planning
B. Kim and L. Shimanuki · 2019
Later among the works it cites.
6-dof graspnet: Variational grasp generation for object manipulation
A. Mousavian, C. Eppner, and D. Fox · 2019
Later among the works it cites.
6-dof grasping for target-driven object manipulation in clutter
Original
A. Murali, A. Mousavian, C. Eppner, C. Paxton, and D. Fox · 2019
Later among the works it cites.
Keto: Learning keypoint representations for tool manipulation
Original
Z. Qin, K. Fang, Y. Zhu, L. Fei-Fei, and S. Savarese · 2019
Later among the works it cites.
Dynamics learning with cascaded variational inference for multi-step manipulation
K. Fang, Y. Zhu, A. Garg, S. Savarese, and L. Fei-Fei · 2019
Later among the works it cites.
Detectron2
Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick · 2019
Later among the works it cites.
Adversarial actor-critic method for task and motion planning problems using planning experience
B. Kim, L. P. Kaelbling, and T. Lozano-Perez · 2019
Later among the works it cites.
Learning quickly to plan quickly using modular meta-learning
T. Rohan Chitnis, Lozano-Pérez and L. P. Kaelbling · 2019
Later among the works it cites.
Analysis and noise modeling of the intel realsense d435 for mobile robots
M. S. Ahn, H. Chae, D. Noh, H. Nam, and D. Hong · 2019
Later among the works it cites.
Fast graph representation learning with PyTorch Geometric
M. Fey and J. E. Lenssen · 2019
Later among the works it cites.
Tactile dexterity: Manipulation primitives with tactile feedback, 2020
F. R. Hogan, J. Ballester, S. Dong, and A. Rodriguez · 2020
Closest in time.
Planar in-hand manipulation via motion cones
N. Chavan-Dafle, R. Holladay, and A. Rodriguez · 2020
Closest in time.
Deep visual reasoning: Learning to predict action sequences for task and motion planning from an initial scene image
Original
D. Driess, J.-S. Ha, and M. Toussaint · 2020
Closest in time.
Learning task-oriented grasping for tool manipulation from simulated self-supervision
K. Fang, Y. Zhu, A. Garg, A. Kurenkov, V. Mehta, L. Fei-Fei, and S. Savarese · 2020
Closest in time.
Learning continuous 3d reconstructions for geometrically aware grasping
Original
M. Van der Merwe, Q. Lu, B. Sundaralingam, M. Matak, and T. Hermans · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…