Fetching the paper…

Reading the bibliography…

2025

ManiAgent: An Agentic Framework for General Robotic Manipulation

Yang, Yi, Gu, Kefan, Wen, Yuqing et al.

Original
PDF
Post on X

Understand

While Vision-Language-Action (VLA) models have demonstrated impressive capabilities in robotic manipulation, their performance in complex reasoning and long-horizon task planning is limited by data scarcity and model capacity.

To address this, we introduce ManiAgent, an agentic architecture for general manipulation tasks that achieves end-to-end output from task descriptions and environmental inputs to robotic manipulation actions.
In this framework, multiple agents involve inter-agent communication to perform environmental perception, sub-task decomposition and action generation, enabling efficient handling of complex manipulation scenarios.
Evaluations show ManiAgent achieves an 86.8% success rate on the SimplerEnv benchmark and 95.8% on real-world pick-and-place tasks, enabling efficient data collection that yields VLA models with performance comparable to those trained on human-annotated datasets.

ManiAgent: An Agentic Framework for General Robotic Manipulation · Around

Built on

A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, et al. , “Rt-1: Robotics Transformer for Real-World Control at Scale.” in Robotics: Science and Systems Conference (RSS) , 2023
2023
Earlier work this paper cites.
H. Wu, Y. Jing, C.-H. Cheang, G. Chen, J. Xu, et al. , “Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation,” in International Conference on Learning Representations , vol. abs/2312.13139, 2023
Original
2023
Earlier work this paper cites.
A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, K. Choromanski, et al. , “Rt-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control,” in Conference on Robot Learning , vol. abs/2307.15818, 2023
Original
2023
Earlier work this paper cites.
W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, et al. , “Voxposer: Composable 3d Value Maps for Robotic Manipulation with Language Models.” in Conference on Robot Learning (CoRL) , 2023, pp. 540–562
2023
Earlier work this paper cites.
I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, et al. , “Progprompt: Generating Situated Robot Task Plans using Large Language Models.” in IEEE International Conference on Robotics and Automation (ICRA) , vol. 47, no. 8, 2023, pp. 11 523–11 530
2023
Earlier work this paper cites.
J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, et al. , “Code as Policies: Language Model Programs for Embodied Control.” in IEEE International Conference on Robotics and Automation (ICRA) , 2023, pp. 9493–9500
2023
Earlier work this paper cites.
K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. D. Reid, et al. , “Sayplan: Grounding Large Language Models using 3d Scene Graphs for Scalable Robot Task Planning.” in Conference on Robot Learning (CoRL) , 2023, pp. 23–72
2023
Earlier work this paper cites.
H. Fang, C. Wang, H. Fang, M. Gou, J. Liu, et al. , “Anygrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains.” IEEE Transactions on Robotics , vol. 39, no. 5, pp. 3929–3945, 2023
2023
Earlier work this paper cites.
Q. Li, Y. Liang, Z. Wang, L. Luo, X. Chen, et al. , “Cogact: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation,” arXiv.org , vol. abs/2411.19650, 2024
Original
2024
Earlier work this paper cites.
K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, et al. , “ π 0 \pi_{0} : A vision-language-action flow model for general robot control,” arXiv preprint arXiv:2410.24164 , 2024
Original
2024
Earlier work this paper cites.
M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, et al. , “Openvla: An Open-Source Vision-Language-Action Model,” in 8th Annual Conference on Robot Learning , vol. abs/2406.09246, 2024
Original
2024
Earlier work this paper cites.

Similar

S. Belkhale, T. Ding, T. Xiao, P. Sermanet, Q. Vuong, et al. , “Rt-H: Action Hierarchies Using Language,” Robotics , vol. abs/2403.01823, 2024
Original
2024
Cited alongside, same era.
X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, et al. , “Evaluating Real-World Robot Manipulation Policies in Simulation,” in RSS 2024 Workshop: Data Generation for Robotics , vol. abs/2405.05941, 2024
Original
2024
Cited alongside, same era.
X. Li, P. Li, M. Liu, D. Wang, J. Liu, et al. , “Towards Generalist Robot Policies: What Matters in Building Vision-Language-Action Models,” arXiv.org , vol. abs/2412.14058, 2024
Original
2024
Cited alongside, same era.
Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, et al. , “Toolllm: Facilitating Large Language Models to Master 16000+ Real-world APIs,” in The Twelfth International Conference on Learning Representations , 2024
2024
Cited alongside, same era.

Then

B. Xiao, H. Wu, W. Xu, X. Dai, H. Hu, et al. , “Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2024
2024
Later among the works it cites.
M. Ahn, D. Dwibedi, C. Finn, M. G. Arenas, K. Gopalakrishnan, et al. , “Autort: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents,” in First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024 , 2024
2024
Later among the works it cites.
K. Bousmalis, G. Vezzani, D. Rao, C. M. Devin, A. X. Lee, et al. , “Robocat: A Self-Improving Generalist Agent for Robotic Manipulation.” Transactions on Machine Learning Research (TMLR) , vol. 2024, 2024
2024
Later among the works it cites.
Y. Wen, K. Gu, H. Liu, Y. Zhao, T. Wang, et al. , “Rosa: Harnessing Robot States for Vision-Language and Action Alignment,” arXiv.org , vol. abs/2506.13679, 2025
Original
2025
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Z. Guo, S. Cheng, H. Wang, S. Liang, Y. Qin, et al. , “Stabletoolbench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models.” in Annual Meeting of the Association for Computational Linguistics (ACL) , 2024, pp. 11 143–11 156

2024

Cited alongside, same era.

S. Yuan, K. Song, J. Chen, X. Tan, Y. Shen, et al. , “Easytool: Enhancing LLM-based Agents with Concise Tool Instruction,” in ICLR 2024 Workshop on Large Language Model (LLM) Agents , vol. abs/2401.06201, 2024

2024

Cited alongside, same era.

W. Huang, C. Wang, Y. Li, R. Zhang, and L. Fei-Fei, “Rekep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation,” in 8th Annual Conference on Robot Learning , 2024

2024

Cited alongside, same era.

J. Duan, W. Yuan, W. Pumacay, Y. R. Wang, K. Ehsani, et al. , “Manipulate-Anything: Automating Real-World Robots using Vision-Language Models,” in RSS 2024 Workshop: Data Generation for Robotics , vol. abs/2406.18915, 2024

2024

Cited alongside, same era.

J. Li, J. Li, J. Chen, Y. Li, S. Wang, et al. , “Evolving Agents: Interactive Simulation of Dynamic and Diverse Human Personalities,” arXiv.org , vol. abs/2404.02718, 2024

2024

Cited alongside, same era.

A. Mei, G.-N. Zhu, H. Zhang, and Z. Gan, “Replanvlm: Replanning Robotic Tasks With Visual Language Models.” IEEE Robotics and Automation Letters , vol. 9, no. 11, pp. 10 201–10 208, 2024

2024

Cited alongside, same era.

H. Singh, R. J. Das, M. Han, P. Nakov, and I. Laptev, “Malmm: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation,” arXiv.org , vol. abs/2411.17636, 2024

2024

Cited alongside, same era.

J. Zhou, K. Ye, J. Liu, T. Ma, Z. Wang, et al. , “Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalization,” arXiv.org , vol. abs/2505.15660, 2025

Original

2025

Closest in time.

S. Han, B. Qiu, Y. Liao, S. Huang, C. Gao, et al. , “Robocerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation,” arXiv.org , vol. abs/2506.06677, 2025

2025

Closest in time.

H. Chen, Z. Song, B. Niu, K. Zhang, L. Ou, et al. , “Toleap: Rethinking Development of Tool Learning with Large Language Models,” arXiv.org , vol. abs/2505.11833, 2025

2025

Closest in time.

W. Tang, J.-H. Pan, Y. Liu, M. Tomizuka, E. L. Li, et al. , “Geomanip: Geometric Constraints as General Interfaces for Robot Manipulation,” arXiv.org , vol. abs/2501.09783, 2025

2025

Closest in time.

G. Yin, Y. Li, Y. Wang, D. Mcconachie, P. Shah, et al. , “Codediffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity,” arXiv.org , vol. abs/2506.16652, 2025

2025

Closest in time.

W. Chen, Z. You, R. Li, y. guan, C. Qian, et al. , “Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence,” in The Thirteenth International Conference on Learning Representations , vol. abs/2407.07061, 2025

2025

Closest in time.

Z. Lou, K. Xu, Z. Zhou, and R. Xiong, “Explorevlm: Closed-Loop Robot Exploration Task Planning with Vision-Language Models,” arXiv , 2025

2025

Closest in time.