Fetching the paper…
Reading the bibliography…
Large Language Models (LLMs) are being explored for applications in scientific research, including their capabilities to synthesize literature, answer research questions, generate research ideas, and even conduct computational experiments.
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis, 2020
Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma · 2009
Earlier work this paper cites.
Principles of high-dimensional data visualization in astronomy
A. A. Goodman · 2012
Earlier work this paper cites.
The NOAO Data Laboratory: a conceptual overview
Michael J. Fitzpatrick, Knut Olsen, Frossie Economou, Elizabeth B. Stobie, T. C. Beers, Mark Dickinson, Patrick Norris, Abi Saha, Robert Seaman, David R. Silva, Robert A. Swaters, Brian Thomas, and Francisco Valdes · 2014
Earlier work this paper cites.
Data Lab-A community science platform
R. Nikutta, M. Fitzpatrick, A. Scott, and B. A. Weaver · 2020
Earlier work this paper cites.
Program synthesis with large language models, 2021
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton · 2021
Earlier work this paper cites.
Evaluating large language models trained on code, 2021
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba · 2021
Earlier work this paper cites.
Jupyter-Enabled Astrophysical Analysis Using Data-Proximate Computing Platforms
Stephanie Juneau, Knut Olsen, Robert Nikutta, Alice Jacques, and Stephen Bailey · 2021
Earlier work this paper cites.
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation, 2022
Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Scott Wen tau Yih, Daniel Fried, Sida Wang, and Tao Yu · 2022
Earlier work this paper cites.
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque · 2022
Earlier work this paper cites.
Beyond Generating Code: Evaluating GPT on a Data Visualization Course
Zhutian Chen, Chenyang Zhang, Qianwen Wang, Jakob Troidl, Simon Warchol, Johanna Beyer, Nils Gehlenborg, and Hanspeter Pfister · 2023
Earlier work this paper cites.
Is GPT-4 a good data analyst?
Liying Cheng, Xingxuan Li, and Lidong Bing · 2023
Earlier work this paper cites.
LIDA: A tool for automatic generation of grammar-agnostic visualizations and infographics using large language models
Victor Dibia · 2023
Earlier work this paper cites.
ChartLlama: A Multimodal LLM for Chart Understanding and Generation, 2023
Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang · 2023
Earlier work this paper cites.
MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation
Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec · 2023
Earlier work this paper cites.
Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang · 2023
Cited alongside, same era.
Paula Maddigan and Teo Susnjak · 2023
Cited alongside, same era.
Execution-based evaluation for open-domain code generation, 2023
Zhiruo Wang, Shuyan Zhou, Daniel Fried, and Graham Neubig · 2023
Cited alongside, same era.
AutoML-GPT: Automatic Machine Learning with GPT
Shujian Zhang, Chengyue Gong, Lemeng Wu, Xingchao Liu, and Mi Zhou · 2023
Cited alongside, same era.
Codebertscore: Evaluating code generation with pretrained models of code, 2023
Berkeley Function Calling Leaderboard
Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, Tianjun Zhang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez · 2024
Later among the works it cites.
MatPlotAgent: Method and evaluation for LLM-based agentic scientific data visualization
Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, and Maosong Sun · 2024
Later among the works it cites.
MLE-bench: Evaluating machine learning agents on machine learning engineering
Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng · 2025
Closest in time.
VisEval: A Benchmark for Data Visualization in the Era of Large Language Models
Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang · 2025
Closest in time.
CodeScientist: End-to-End Semi-Automated Scientific Discovery with Code-based Experimentation
Peter Jansen, Oyvind Tafjord, Marissa Radensky, Pao Siangliulue, Tom Hope, Bhavana Dalvi, Bodhisattwa Prasad Majumder, Daniel S. Weld, and Peter Clark · 2025
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Shuyan Zhou, Uri Alon, Sumit Agarwal, and Graham Neubig · 2023
Cited alongside, same era.
Try Deep Research and our new experimental model in Gemini, your AI assistant, 2024
Google · 2024
Cited alongside, same era.
Visualization generation with large language models: An evaluation, 2024
Guozheng Li, Xinyu Wang, Gerile Aodeng, Shunyuan Zheng, Yu Zhang, Chuangxin Ou, Song Wang, and Chi Harold Liu · 2024
Cited alongside, same era.
The AI scientist: Towards fully automated open-ended scientific discovery
Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha · 2024
Cited alongside, same era.
Deep research system card, 2024
OpenAI · 2024
Cited alongside, same era.
Luca Podo, Muhammad Ishmal, and Marco Angelini · 2024
Cited alongside, same era.
Can LLMs generate novel research ideas? A large-scale human study with 100+ NLP researchers
Chenglei Si, Diyi Yang, and Tatsunori Hashimoto · 2024
Cited alongside, same era.
SciCode: A research coding benchmark curated by scientists
Minyang Tian, Luyu Gao, Shizhuo Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, et al · 2024
Cited alongside, same era.
Closest in time.
How Good is ChatGPT in Giving Advice on Your Visualization Design?, 2025
Nam Wook Kim, Yongsu Ahn, Grace Myers, and Benjamin Bach · 2025
Closest in time.
Gravity-Bench-v1: A Benchmark on Gravitational Physics Discovery for Agents, 2025
Nolan Koblischke, Hyunseok Jang, Kristen Menou, and Mohamad Ali-Dib · 2025
Closest in time.
The Promises and Pitfalls of AI Scientists
Emmy Liu · 2025
Closest in time.
PaperBench: Evaluating AI’s Ability to Replicate AI Research
Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, et al · 2025
Closest in time.
CSR-bench: Benchmarking LLM agents in deployment of computer science research repositories
Yijia Xiao, Runhui Wang, Luyang Kong, Davor Golac, and Wei Wang · 2025
Closest in time.
ChartCoder: Advancing multimodal large language model for chart-to-code generation
Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, and Maosong Sun · 2025
Closest in time.
DomainCQA: Crafting Expert-Level QA from Domain-Specific Charts
Ling Zhong, Yujing Lu, Jing Yang, Weiming Li, Peng Wei, Yongheng Wang, Manni Duan, and Qing Zhang · 2025
Closest in time.
BigCodeBench: Benchmarking code generation with diverse function calls and complex instructions
Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al · 2025
Closest in time.