Fetching the paper…
Reading the bibliography…
We introduce MLE-Dojo, a Gym-style framework for systematically reinforcement learning, evaluating, and improving autonomous large language model (LLM) agents in iterative machine learning engineering (MLE) workflows.
Program synthesis with large language models
J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al · 2021
Earlier work this paper cites.
Evaluating large language models trained on code
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al · 2021
Earlier work this paper cites.
Measuring coding challenge competence with APPS
D. Hendrycks, S. Basart, S. Kadavath, M. Mazeika, A. Arora, E. Guo, C. Burns, S. Puranik, H. He, D. Song, and J. Steinhardt · 2021
Earlier work this paper cites.
Competition-level code generation with alphacode
Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago, et al · 2022
Earlier work this paper cites.
Swe-bench: Can language models resolve real-world github issues?
C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan · 2023
Earlier work this paper cites.
Ds-1000: A natural and reliable benchmark for data science code generation
Y. Lai, C. Li, Y. Wang, T. Zhang, R. Zhong, L. Zettlemoyer, W.-t. Yih, D. Fried, S. Wang, and T. Yu · 2023
Earlier work this paper cites.
Natural language to code generation in interactive data science notebooks
P. Yin, W.-D. Li, K. Xiao, A. Rao, Y. Wen, K. Shi, J. Howland, P. Bailey, M. Catasta, H. Michalewski, et al · 2023
Earlier work this paper cites.
Mle-bench: Evaluating machine learning agents on machine learning engineering
J. S. Chan, N. Chowdhury, O. Jaffe, J. Aung, D. Sherburn, E. Mays, G. Starace, K. Liu, L. Maksin, T. Patwardhan, et al · 2024
Earlier work this paper cites.
Sela: Tree-search enhanced llm agents for automated machine learning
Y. Chi, Y. Lin, S. Hong, D. Pan, Y. Fei, G. Mei, B. Liu, T. Pang, J. Kwok, C. Zhang, et al · 2024
Earlier work this paper cites.
Chatbot arena: An open platform for evaluating llms by human preference
W.-L. Chiang, L. Zheng, Y. Sheng, A. N. Angelopoulos, T. Li, D. Li, B. Zhu, H. Zhang, M. Jordan, J. E. Gonzalez, et al · 2024
Earlier work this paper cites.
Workarena: How capable are web agents at solving common knowledge work tasks?
A. Drouin, M. Gasse, M. Caccia, I. H. Laradji, M. D. Verme, T. Marty, D. Vazquez, N. Chapados, and A. Lacoste · 2024
Earlier work this paper cites.
Magentic-one: A generalist multi-agent system for solving complex tasks
A. Fourney, G. Bansal, H. Mozannar, C. Tan, E. Salinas, F. Niedtner, G. Proebsting, G. Bassman, J. Gerrits, J. Alber, et al · 2024
Earlier work this paper cites.
Large language models orchestrating structured reasoning achieve kaggle grandmaster level
A. Grosnit, A. Maraval, J. Doran, G. Paolo, A. Thomas, R. S. H. N. Beevi, J. Gonzalez, K. Khandelwal, I. Iacobacci, A. Benechehab, et al · 2024
Earlier work this paper cites.
DS-agent: Automated data science by empowering large language models with case-based reasoning
S. Guo, C. Deng, Y. Wen, H. Chen, Y. Chang, and J. Wang · 2024
Cited alongside, same era.
Large language models for automated data science: Introducing caafe for context-aware automated feature engineering
N. Hollmann, S. Müller, and F. Hutter · 2024
Cited alongside, same era.
Data interpreter: An llm agent for data science
S. Hong, Y. Lin, B. Liu, B. Liu, B. Wu, C. Zhang, C. Wei, D. Li, J. Chen, J. Zhang, et al · 2024
Cited alongside, same era.
MLAgentbench: Evaluating language agents on machine learning experimentation
Q. Huang, J. Vora, P. Liang, and J. Leskovec · 2024
Cited alongside, same era.
A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al · 2024
Benchmarking data science agents
Y. Zhang, Q. Jiang, X. Han, N. Chen, Y. Yang, and K. Ren · 2024
Later among the works it cites.
Pybench: Evaluating llm agent on various real-world coding tasks
Y. Zhang, Y. Pan, Y. Wang, and J. Cai · 2024
Later among the works it cites.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025
DeepSeek-AI · 2025
Closest in time.
Gemini 2.0 flash: Our powerful workhorse model with low latency and enhanced performance, built to power agentic experiences
Google · 2025
Closest in time.
Gemini 2.0 pro: Our best model yet for coding performance and complex prompts
Google · 2025
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Autokaggle: A multi-agent framework for autonomous data science competitions
Z. Li, Q. Zang, D. Ma, J. Guo, T. Zheng, M. Liu, X. Niu, Y. Wang, J. Yang, J. Liu, et al · 2024
Cited alongside, same era.
A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al · 2024
Cited alongside, same era.
Training software engineering agents and verifiers with swe-gym
J. Pan, X. Wang, G. Neubig, N. Jaitly, H. Ji, A. Suhr, and Y. Zhang · 2024
Cited alongside, same era.
Collaborative gym: A framework for enabling and evaluating human-agent collaboration
Y. Shao, V. Samuel, Y. Jiang, J. Yang, and D. Yang · 2024
Cited alongside, same era.
Automl-agent: A multi-agent llm framework for full-pipeline automl
P. Trirat, W. Jeong, and S. J. Hwang · 2024
Cited alongside, same era.
Openhands: An open platform for ai software developers as generalist agents
X. Wang, B. Li, Y. Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y. Song, B. Li, J. Singh, et al · 2024
Cited alongside, same era.
Swe-agent: Agent-computer interfaces enable automated software engineering
J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. R. Narasimhan, and O. Press · 2024
Cited alongside, same era.
Gemini 2.5: Our most intelligent ai model
Google · 2025
Closest in time.
Aide: Ai-driven exploration in the space of code
Z. Jiang, D. Schmidt, D. Srikanth, D. Xu, I. Kaplan, D. Jacenko, and Y. Wu · 2025
Closest in time.
DSBench: How far are data science agents from becoming data science experts?
L. Jing, Z. Huang, X. Wang, W. Yao, W. Yu, K. Ma, H. Zhang, X. Du, and D. Yu · 2025
Closest in time.
Mlgym: A new framework and benchmark for advancing ai research agents, 2025
D. Nathani, L. Madaan, N. Roberts, N. Bashlykov, A. Menon, V. Moens, A. Budhiraja, D. Magka, V. Vorotilov, G. Chaurasia, D. Hupkes, R. S. Cabral, T. Shavrina, J. Foerster, Y. Bachrach, W. Y. Wang, and R. Raileanu · 2025
Closest in time.
Openai o3-mini: Pushing the frontier of cost-effective reasoning
OpenAI · 2025
Closest in time.
Agentrxiv: Towards collaborative autonomous research
S. Schmidgall and M. Moor · 2025
Closest in time.
Multi-swe-bench: A multilingual benchmark for issue resolving
D. Zan, Z. Huang, W. Liu, H. Chen, L. Zhang, S. Xin, L. Chen, Q. Liu, X. Zhong, A. Li, et al · 2025
Closest in time.