VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generation

Y. Jiang, S. Moseson, and A. Saxena, “Efficient grasping from rgbd images: Learning using a new rectangle representation,” in 2011 IEEE International conference on robotics and automation . IEEE, 2011, pp. 3304–3311

2011

Earlier work this paper cites.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014

Original

2014

Earlier work this paper cites.

J. Redmon and A. Angelova, “Real-time grasp detection using convolutional neural networks,” in 2015 IEEE international conference on robotics and automation (ICRA) . IEEE, 2015, pp. 1316–1322

2015

Earlier work this paper cites.

S. Kumra and C. Kanan, “Robotic grasp detection using deep convolutional neural networks,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2017, pp. 769–776

2017

Earlier work this paper cites.

P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, et al. , “Mixed precision training,” arXiv preprint arXiv:1710.03740 , 2017

Original

2017

Earlier work this paper cites.

D. Morrison, P. Corke, and J. Leitner, “Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach,” Robotics: Science and Systems XIV , pp. 1–10, 2018

2018

Earlier work this paper cites.

A. Mousavian, C. Eppner, and D. Fox, “6-dof graspnet: Variational grasp generation for object manipulation,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 2901–2910

2019

Earlier work this paper cites.

D. Kalamkar, D. Mudigere, N. Mellempudi, D. Das, K. Banerjee, S. Avancha, D. T. Vooturi, N. Jammalamadaka, J. Huang, H. Yuen, et al. , “A study of bfloat16 for deep learning training,” arXiv preprint arXiv:1905.12322 , 2019

Original

2019

Earlier work this paper cites.

S. Kumra, S. Joshi, and F. Sahin, “Antipodal robotic grasping using generative residual convolutional neural network,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2020, pp. 9626–9633

2020

Earlier work this paper cites.

H.-S. Fang, C. Wang, M. Gou, and C. Lu, “Graspnet-1billion: A large-scale benchmark for general object grasping,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 11 444–11 453

2020

Earlier work this paper cites.

S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He, “Zero: Memory optimizations toward training trillion parameter models,” in SC20: International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 2020, pp. 1–16

2020

Earlier work this paper cites.

S. Ainetter and F. Fraundorfer, “End-to-end trainable deep neural network for robotic grasp detection and semantic segmentation from rgb,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 13 452–13 458

2021

Earlier work this paper cites.

M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox, “Contact-graspnet: Efficient 6-dof grasp generation in cluttered scenes,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 13 438–13 444

2021

Earlier work this paper cites.

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PmLR, 2021, pp. 8748–8763

2021

Earlier work this paper cites.

Y. Lu, B. Deng, Z. Wang, P. Zhi, Y. Li, and S. Wang, “Hybrid physical metric for 6-dof grasp pose detection,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 8238–8244

2022

Earlier work this paper cites.

K. Xu, S. Zhao, Z. Zhou, Z. Li, H. Pi, Y. Zhu, Y. Wang, and R. Xiong, “A joint modeling of vision-language-action for target-oriented grasping in clutter,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 11 597–11 604

2023

Cited alongside, same era.

C. Tang, D. Huang, W. Ge, W. Liu, and H. Zhang, “Graspgpt: Leveraging semantic knowledge from a large language model for task-oriented grasping,” IEEE Robotics and Automation Letters , vol. 8, no. 11, pp. 7551–7558, 2023

2023

Cited alongside, same era.

H.-S. Fang, C. Wang, H. Fang, M. Gou, J. Liu, H. Yan, W. Liu, Y. Xie, and C. Lu, “Anygrasp: Robust and efficient grasp perception in spatial and temporal domains,” IEEE Transactions on Robotics , vol. 39, no. 5, pp. 3929–3945, 2023

2023

Cited alongside, same era.

Y. Lu, Y. Fan, B. Deng, F. Liu, Y. Li, and S. Wang, “Vl-grasp: a 6-dof interactive grasp policy for language-oriented objects in cluttered indoor scenes,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 976–983

H. Shao, S. Qian, H. Xiao, G. Song, Z. Zong, L. Wang, Y. Liu, and H. Li, “Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning,” Advances in Neural Information Processing Systems , vol. 37, pp. 8612–8642, 2024

2024

Later among the works it cites.

J. Zheng, J. Li, S. Cheng, Y. Zheng, J. Li, J. Liu, Y. Liu, J. Liu, and X. Zhan, “Instruction-guided visual masking,” Advances in neural information processing systems , vol. 37, pp. 126 004–126 031, 2024

2024

Later among the works it cites.

A. D. Vuong, M. N. Vu, H. Le, B. Huang, H. T. T. Binh, T. Vo, A. Kugi, and A. Nguyen, “Grasp-anything: Large-scale grasp dataset from foundation models,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 14 030–14 037

2024

Later among the works it cites.

A. Steiner, A. S. Pinto, M. Tschannen, D. Keysers, X. Wang, Y. Bitton, A. Gritsenko, M. Minderer, A. Sherbondy, S. Long, et al. , “Paligemma 2: A family of versatile vlms for transfer,” arXiv preprint arXiv:2412.03555 , 2024