Fetching the paper…

Reading the bibliography…

2023

CoPAL: Corrective Planning of Robot Actions with Large Language Models

Joublin, Frank, Ceravola, Antonello, Smirnov, Pavel et al.

Original
PDF
Post on X

Understand

In the pursuit of fully autonomous robotic systems capable of taking over tasks traditionally performed by humans, the complexity of open-world environments poses a considerable challenge.

Addressing this imperative, this study contributes to the field of Large Language Models (LLMs) applied to task and motion planning for robots.
We propose a system architecture that orchestrates a seamless interplay between multiple cognitive levels, encompassing reasoning, planning, and motion generation.
At its core lies a novel replanning strategy that handles physically grounded, logical, and semantic errors in the generated plans.

CoPAL: Corrective Planning of Robot Actions with Large Language Models · Around

Built on

D. McDermott, M. Ghallab, A. Howe, C. Knoblock, A. Ram, M. Veloso, D. Weld, and D. Wilkins, “PDDL - The Planning Domain Definition Language,” The AIPS-98 Planning Competition Committee , 1998
1998
Earlier work this paper cites.
M. Gienger, M. Toussaint, and C. Goerick, “Whole-body motion planning–building blocks for intelligent systems,” Motion Planning for Humanoid Robots , p. 67, 2010
2010
Earlier work this paper cites.
P. Haslum, N. Lipovetzky, D. Magazzeni, C. Muise, R. Brachman, F. Rossi, and P. Stone, An introduction to the Planning Domain Definition Language . Springer, 2019, vol. 13
2019
Earlier work this paper cites.
E. Karpas and D. Magazzeni, “Automated planning for robotics,” Annual Review of Control, Robotics, and Autonomous Systems , 2020
2020
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , 2020
2020
Earlier work this paper cites.
B. Sendhoff and H. Wersing, “Cooperative intelligence-a humane perspective,” in 2020 IEEE International Conference on Human-Machine Systems (ICHMS) . IEEE, 2020, pp. 1–6
2020
Earlier work this paper cites.
C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, T. Silver, L. P. Kaelbling, and T. Lozano-Pérez, “Integrated task and motion planning,” Annual Review of Control, Robotics, and Autonomous Systems , 2021
2021
Earlier work this paper cites.
S. Patra, J. Mason, M. Ghallab, D. Nau, and P. Traverso, “Deliberative acting, planning and learning with hierarchical operational models,” Artificial Intelligence , vol. 299, p. 103523, 2021
2021
Earlier work this paper cites.
S. Li, X. Puig, C. Paxton, Y. Du, C. Wang, L. Fan, T. Chen, D.-A. Huang, E. Akyürek, A. Anandkumar et al. , “Pre-trained language models for interactive decision-making,” Advances in Neural Information Processing Systems , 2022
2022
Earlier work this paper cites.
W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” International Conference on Machine Learning , 2022
2022
Earlier work this paper cites.
K. Valmeekam, A. Olmo, S. Sreedharan, and S. Kambhampati, “Large Language Models still can’t plan (A Benchmark for LLMs on Planning and Reasoning about Change,” arXiv preprint arXiv:2206.10498 , 2022
Original
2022
Earlier work this paper cites.
S. S. Raman, V. Cohen, E. Rosen, I. Idrees, D. Paulius, and S. Tellex, “Planning with Large Language Models via corrective re-prompting,” arXiv preprint arXiv:2211.09935 , 2022
Original
2022
Earlier work this paper cites.
W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar et al. , “Inner Monologue: Embodied reasoning through planning with language models,” arXiv preprint arXiv:2207.05608 , 2022
Original
2022
Earlier work this paper cites.
R. I. Brafman, D. Tolpin, and O. Wertheim, “Probabilistic programs as an action description language,” AAAI Conference on Artificial Intelligence , 2023
2023
Earlier work this paper cites.

Similar

D. Tanneberg and M. Gienger, “Learning type-generalized actions for symbolic planning,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2023
2023
Cited alongside, same era.
S. Vemprala, R. Bonatti, A. Bucker, and A. Kapoor, “ChatGPT for robotics: Design principles and model abilities,” Microsoft, Tech. Rep., 2023
2023
Cited alongside, same era.
N. Wake, A. Kanehira, K. Sasabuchi, J. Takamatsu, and K. Ikeuchi, “ChatGPT empowered long-step robot control in various environments: A case application,” IEEE Access , 2023
2023
Cited alongside, same era.
G. DeepMind, “Demonstrating Large Language Models on robots,” Robotics: Science and Systems (RSS) Demo Track , 2023
2023
Cited alongside, same era.
T. Yoneda, J. Fang, P. Li, H. Zhang, T. Jiang, S. Lin, B. Picker, D. Yunis, H. Mei, and M. R. Walter, “Statler: State-maintaining language models for embodied reasoning,” arXiv preprint arXiv:2306.17840 , 2023

Then

J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, B. Yin, and X. Hu, “Harnessing the power of LLMs in practice: A survey on ChatGPT and beyond,” arXiv preprint arXiv:2304.13712 , 2023
Original
2023
Closest in time.
F. Joublin, A. Ceravola, J. Deigmoeller, M. Gienger, M. Franzius, and J. Eggert, “A glimpse in ChatGPT capabilities and its impact for AI research,” arXiv preprint arXiv:2305.06087 , 2023
Original
2023
Closest in time.
S. Yang, O. Nachum, Y. Du, J. Wei, P. Abbeel, and D. Schuurmans, “Foundation models for decision making: Problems, methods, and opportunities,” arXiv preprint arXiv:2303.04129 , 2023
Original
2023
Closest in time.
S. Kambhampati, “Can LLMs really reason and plan?” Blog at Communications of the ACM , 2023. [Online]. Available: https://cacm.acm.org/blogs/blog-cacm/276268-can-llms-really-reason-and-plan/fulltext
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2023

Cited alongside, same era.

J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser, “Tidybot: Personalized robot assistance with large language models,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2023

2023

Cited alongside, same era.

M. Gramopadhye and D. Szafir, “Generating executable action plans with environmentally-aware language models,” arXiv preprint arXiv:2210.04964 , 2023

2023

Cited alongside, same era.

Y. Ding, X. Zhang, S. Amiri, N. Cao, H. Yang, A. Kaminski, C. Esselink, and S. Zhang, “Integrating action knowledge and LLMs for task planning and situation handling in open worlds,” Autonomous Robots (2023) , 2023

2023

Cited alongside, same era.

D. Driess, F. Xia, M. S. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huang, Y. Chebotar, P. Sermanet, D. Duckworth, S. Levine, V. Vanhoucke, K. Hausman, M. Toussaint, K. Greff, A. Zeng, I. Mordatch, and P. Florence, “Palm-e: An embodied multimodal language model,” International Conference on Machine Learning , 2023

2023

Cited alongside, same era.

J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for embodied control,” IEEE International Conference on Robotics and Automation (ICRA) , 2023

2023

Cited alongside, same era.

Y. Ding, X. Zhang, C. Paxton, and S. Zhang, “Task and motion planning with large language models for object rearrangement,” arXiv preprint arXiv:2303.06247 , 2023

2023

Cited alongside, same era.

Z. Zhao, W. S. Lee, and D. Hsu, “Large Language Models as commonsense knowledge for large-scale task planning,” arXiv preprint arXiv:2305.14078 , 2023

2023

Cited alongside, same era.

Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, and H. Soh, “Translating natural language to planning goals with Large-language models,” arXiv preprint arXiv:2302.05128 , 2023

Original

2023

Closest in time.

I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situated robot task plans using Large Language Models,” IEEE International Conference on Robotics and Automation (ICRA) , 2023

2023

Closest in time.

Z. Wang, S. Cai, A. Liu, X. Ma, and Y. Liang, “Describe, explain, plan and select: Interactive planning with Large Language Models enables open-world multi-task agents,” arXiv preprint arXiv:2302.01560 , 2023

2023

Closest in time.

B. Liu, Y. Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone, “LLM+P: Empowering Large Language Models with optimal planning proficiency,” arXiv preprint arXiv:2304.11477 , 2023

2023

Closest in time.

K. Valmeekam, M. Marquez, S. Sreedharan, and S. Kambhampati, “On the planning abilities of Large Language Models – A critical investigation,” arXiv preprint arXiv:2305.15771 , 2023

2023

Closest in time.

L. Guan, K. Valmeekam, S. Sreedharan, and S. Kambhampati, “Leveraging pre-trained Large Language Models to construct and utilize world models for model-based task planning,” arXiv preprint arXiv:2305.14909 , 2023

2023

Closest in time.

B. Ichter, A. Brohan, Y. Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho et al. , “Do as I can, not as I say: Grounding language in robotic affordances,” Conference on Robot Learning , 2023

2023

Closest in time.

K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg, “Text2motion: From natural language instructions to feasible plans,” arXiv preprint arXiv:2303.12153 , 2023

2023

Closest in time.

K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suenderhauf, “SayPlan: Grounding Large Language Models using 3D scene graphs for scalable task planning,” arXiv preprint arXiv:2307.06135 , 2023

2023

Closest in time.

H. Bharadhwaj, J. Vakil, M. Sharma, A. Gupta, S. Tulsiani, and V. Kumar, “Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking,” arXiv preprint arXiv:2309.01918 , 2023

2023

Closest in time.