2019

Joint Inference of Kinematic and Force Trajectories with Visuo-Tactile Sensing

Lambert, Alexander, Mukadam, Mustafa, Sundaralingam, Balakumar et al.

Understand

To perform complex tasks, robots must be able to interact with and manipulate their surroundings.

One of the key challenges in accomplishing this is robust state estimation during physical interactions, where the state involves not only the robot and the object being manipulated, but also the state of the contact itself.
In this work, within the context of planar pushing, we extend previous inference-based approaches to state estimation in several ways.
We estimate the robot, object, and the contact state on multiple manipulation platforms configured with a vision-based articulated model tracker, and either a biomimetic tactile sensor or a force-torque sensor.

Built on

M. T. Mason, “Mechanics and planning of manipulator pushing operations,” The International Journal of Robotics Research , vol. 5, no. 3, pp. 53–71, 1986
1986
Earlier work this paper cites.
S. H. Lee and M. Cutkosky, “Fixture planning with friction,” Journal of Engineering for Industry , vol. 113, no. 3, pp. 320–327, 1991
1991
Earlier work this paper cites.
K. M. Lynch, H. Maekawa, and K. Tanie, “Manipulation and active sensing by pushing using tactile feedback.” in IROS , 1992, pp. 416–421
1992
Earlier work this paper cites.
M. Montemerlo, S. Thrun, D. Koller, B. Wegbreit, et al. , “Fastslam: A factored solution to the simultaneous localization and mapping problem,” Aaai/iaai , vol. 593598, 2002
2002
Earlier work this paper cites.
S. Thrun and M. Montemerlo, “The graph slam algorithm with applications to large-scale mapping of urban structures,” The International Journal of Robotics Research , vol. 25, no. 5-6, pp. 403–429, 2006
2006
Earlier work this paper cites.
F. Dellaert and M. Kaess, “Square root SAM: Simultaneous localization and mapping via square root information smoothing,” The International Journal of Robotics Research , vol. 25, no. 12, pp. 1181–1203, 2006
2006
Earlier work this paper cites.
L. Zhang and J. C. Trinkle, “The application of particle filtering to grasping acquisition with visual occlusion and tactile sensing,” in Robotics and automation (ICRA), 2012 IEEE international conference on . IEEE, 2012, pp. 3805–3812
2012
Earlier work this paper cites.
F. Dellaert, “Factor graphs and gtsam: A hands-on introduction,” Georgia Institute of Technology, Tech. Rep., 2012
2012
Earlier work this paper cites.

Similar

M. Kaess, H. Johannsson, R. Roberts, V. Ila, J. J. Leonard, and F. Dellaert, “isam2: Incremental smoothing and mapping using the bayes tree,” The International Journal of Robotics Research , vol. 31, no. 2, pp. 216–235, 2012
2012
Cited alongside, same era.
J. Bimbo, L. D. Seneviratne, K. Althoefer, and H. Liu, “Combining touch and vision for the estimation of an object’s pose during manipulation,” in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on . IEEE, 2013, pp. 4021–4026
2013
Cited alongside, same era.
M. Chalon, J. Reinecke, and M. Pfanne, “Online in-hand object localization,” in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on . IEEE, 2013, pp. 2977–2984
2013
Cited alongside, same era.
G. E. Loeb, “Estimating point of contact, force and torque in a biomimetic tactile sensor with deformable skin,” 2013
2013
Cited alongside, same era.
T. Schmidt, R. A. Newcombe, and D. Fox, “DART: Dense articulated real-time tracking.” in Robotics: Science and Systems , vol. 2, no. 1, 2014
2014
Cited alongside, same era.
T. Schmidt, K. Hertkorn, R. Newcombe, Z. Marton, M. Suppa, and D. Fox, “Depth-based tracking with physical constraints for robot manipulation,” in Robotics and Automation (ICRA), 2015 IEEE International Conference on . IEEE, 2015, pp. 119–126
2015
Cited alongside, same era.
K.-T. Yu, J. Leonard, and A. Rodriguez, “Shape and pose recovery from planar pushing,” in Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on . IEEE, 2015, pp. 1208–1215
2015
Cited alongside, same era.
F. R. Hogan and A. Rodriguez, “Feedback control of the pusher-slider system: A story of hybrid and underactuated contact dynamics,” arXiv preprint arXiv:1611.08268 , 2016
Original
2016
Cited alongside, same era.

Then

K.-T. Yu, M. Bauza, N. Fazeli, and A. Rodriguez, “More than a million ways to be pushed. a high-fidelity experimental dataset of planar pushing,” in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on . IEEE, 2016, pp. 30–37
2016
Later among the works it cites.
M. Mukadam, J. Dong, F. Dellaert, and B. Boots, “Simultaneous trajectory estimation and planning via probabilistic inference,” in Proceedings of Robotics: Science and Systems (RSS) , 2017
2017
Later among the works it cites.
M. Mukadam, J. Dong, X. Yan, F. Dellaert, and B. Boots, “Continuous-time Gaussian process motion planning via probabilistic inference,” The International Journal of Robotics Research (IJRR) , 2018
2018
Later among the works it cites.
K.-T. Yu and A. Rodriguez, “Realtime state estimation with tactile and visual sensing. application to planar manipulation,” in 2018 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2018, pp. 7778–7785
2018
Later among the works it cites.
——, “Realtime state estimation with tactile and visual sensing for inserting a suction-held object,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 1628–1635
2018
Later among the works it cites.
A. Lambert, A. Shaban, A. Raj, Z. Liu, and B. Boots, “Deep forward and inverse perceptual models for tracking and prediction,” in 2018 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2018, pp. 675–682
2018
Later among the works it cites.
B. Sundaralingam, A. Lambert, A. Handa, B. Boots, T. Hermans, S. Birchfield, N. Ratliff, and D. Fox, “Robust learning of tactile force estimation through robot interaction,” arXiv preprint arXiv:1810.06187 , 2018
Original
2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…