AlignBot: Aligning VLM-powered Customized Task Planning with User Reminders Through Fine-Tuning for Household Robots

K. Sparck Jones, “A statistical interpretation of term specificity and its application in retrieval,” Journal of documentation , vol. 28, no. 1, pp. 11–21, 1972

1972

Earlier work this paper cites.

Y.-q. Jiang, S.-q. Zhang, P. Khandelwal, and P. Stone, “Task planning in robotics: an empirical comparison of pddl-and asp-based systems,” Frontiers of Information Technology & Electronic Engineering , vol. 20, pp. 363–373, 2019

2019

Earlier work this paper cites.

E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations , 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9

2022

Earlier work this paper cites.

I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situated robot task plans using large language models,” 2022. [Online]. Available: https://arxiv.org/abs/2209.11302

Original

2022

Earlier work this paper cites.

Y. Ding, X. Zhang, S. Amiri, N. Cao, H. Yang, A. Kaminski, C. Esselink, and S. Zhang, “Integrating action knowledge and llms for task planning and situation handling in open worlds,” Autonomous Robots , vol. 47, no. 8, pp. 981–997, 2023

2023

Earlier work this paper cites.

J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser, “Tidybot: Personalized robot assistance with large language models,” Autonomous Robots , vol. 47, no. 8, pp. 1087–1102, 2023

2023

Earlier work this paper cites.

M. Zhang, G. Tian, Y. Cui, Y. Zhang, and Z. Xia, “Hierarchical semantic knowledge-based object search method for household robots,” IEEE Transactions on Emerging Topics in Computational Intelligence , 2023

2023

Earlier work this paper cites.

D. Driess, F. Xia, M. S. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huang, Y. Chebotar, P. Sermanet, D. Duckworth, S. Levine, V. Vanhoucke, K. Hausman, M. Toussaint, K. Greff, A. Zeng, I. Mordatch, and P. Florence, “Palm-e: An embodied multimodal language model,” 2023. [Online]. Available: https://arxiv.org/abs/2303.03378

Original

2023

Earlier work this paper cites.

S. Vemprala, R. Bonatti, A. Bucker, and A. Kapoor, “Chatgpt for robotics: Design principles and model abilities,” 2023. [Online]. Available: https://arxiv.org/abs/2306.17582

Original

2023

Earlier work this paper cites.

K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg, “Text2motion: from natural language instructions to feasible plans,” Autonomous Robots , vol. 47, no. 8, p. 1345–1365, Nov. 2023. [Online]. Available: http://dx.doi.org/10.1007/s10514-023-10131-7

2023

Earlier work this paper cites.

N. Wake, A. Kanehira, K. Sasabuchi, J. Takamatsu, and K. Ikeuchi, “Chatgpt empowered long-step robot control in various environments: A case application,” IEEE Access , vol. 11, p. 95060–95078, 2023. [Online]. Available: http://dx.doi.org/10.1109/ACCESS.2023.3310935

2023

Earlier work this paper cites.

D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al. , “Palm-e: An embodied multimodal language model,” arXiv preprint arXiv:2303.03378 , 2023

Original

2023

Earlier work this paper cites.

D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” arXiv preprint arXiv:2304.10592 , 2023

Original

2023

Earlier work this paper cites.

A. Z. Ren, A. Dixit, A. Bodrova, S. Singh, S. Tu, N. Brown, P. Xu, L. Takayama, F. Xia, J. Varley, Z. Xu, D. Sadigh, A. Zeng, and A. Majumdar, “Robots that ask for help: Uncertainty alignment for large language model planners,” 2023. [Online]. Available: https://arxiv.org/abs/2307.01928

Original

2023

Earlier work this paper cites.

Z. Liu, A. Bahety, and S. Song, “Reflect: Summarizing robot experiences for failure explanation and correction,” 2023. [Online]. Available: https://arxiv.org/abs/2306.15724

Original

2023

Earlier work this paper cites.

Y. Hu, F. Lin, T. Zhang, L. Yi, and Y. Gao, “Look before you leap: Unveiling the power of gpt-4v in robotic vision-language planning,” arXiv preprint arXiv:2311.17842 , 2023

Original

2023

Earlier work this paper cites.

H. Liu, C. Li, Y. Li, and Y. J. Lee, “Improved baselines with visual instruction tuning,” 2023

2023

Cited alongside, same era.

H. Ratnayake and C. Wang, “A prompting framework to enhance language model output,” in Australasian Joint Conference on Artificial Intelligence . Springer, 2023, pp. 66–81

2023

Cited alongside, same era.

F. Joublin, A. Ceravola, P. Smirnov, F. Ocker, J. Deigmoeller, A. Belardinelli, C. Wang, S. Hasler, D. Tanneberg, and M. Gienger, “Copal: Corrective planning of robot actions with large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2310.07263

Original

2023

Cited alongside, same era.

H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, “Llama 2: Open foundation and fine-tuned chat models,” 2023. [Online]. Available: https://arxiv.org/abs/2307.09288

A. Mei, J. Wang, G.-N. Zhu, and Z. Gan, “Gamevlm: A decision-making framework for robotic task planning based on visual language models and zero-sum games,” arXiv preprint arXiv:2405.13751 , 2024

Original

2024

Closest in time.

R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” 2024. [Online]. Available: https://arxiv.org/abs/2305.18290

Original

2024

Closest in time.

Y. Kim, D. Kim, J. Choi, J. Park, N. Oh, and D. Park, “A survey on integration of large language models with intelligent robots,” Intelligent Service Robotics , Aug. 2024. [Online]. Available: http://dx.doi.org/10.1007/s11370-024-00550-5

2024

Closest in time.

Y. Liu, L. Palmieri, S. Koch, I. Georgievski, and M. Aiello, “Towards human awareness in robot task planning with large language models,” 2024. [Online]. Available: https://arxiv.org/abs/2404.11267

Original

2024

Closest in time.