Fetching the paper…
Reading the bibliography…
Synthetic verification techniques such as generating test cases and reward modelling are common ways to enhance the coding capabilities of large language models (LLM) beyond predefined tests.
Program synthesis with large language models, 2021
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton · 2021
Earlier work this paper cites.
Evaluating large language models trained on code, 2021
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba · 2021
Earlier work this paper cites.
Codet: Code generation with generated tests, 2022
Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen · 2022
Earlier work this paper cites.
Competition-level code generation with alphacode
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals · 2022
Earlier work this paper cites.
An empirical evaluation of using large language models for automated unit test generation
Max Schäfer, Sarah Nadi, Aryaz Eghbali, and Frank Tip · 2023
Earlier work this paper cites.
Algo: Synthesizing algorithmic programs with llm-generated oracle verifiers, 2023
Kexun Zhang, Danqing Wang, Jingtao Xia, William Yang Wang, and Lei Li · 2023
Earlier work this paper cites.
Tdd-bench verified: Can llms generate tests for issues before they get resolved?, 2024
Toufique Ahmed, Martin Hirzel, Rangeet Pan, Avraham Shinnar, and Saurabh Sinha · 2024
Earlier work this paper cites.
The llama 3 herd of models, 2024
Meta AI · 2024
Earlier work this paper cites.
Rethinking the influence of source code on test case generation, 2024
Dong Huang, Jie M. Zhang, Mingzhe Du, Mark Harman, and Heming Cui · 2024
Earlier work this paper cites.
Qwen2.5-coder technical report, 2024
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin · 2024
Earlier work this paper cites.
Rewardbench: Evaluating reward models for language modeling, 2024
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi · 2024
Earlier work this paper cites.
Large language models as test case generators: Performance evaluation and enhancement, 2024
Kefan Li and Yuan Yuan · 2024
Cited alongside, same era.
Scattered forest search: Smarter code space exploration with llms, 2024
Jonathan Light, Yue Wu, Yiyou Sun, Wenchao Yu, Yanchi liu, Xujiang Zhao, Ziniu Hu, Haifeng Chen, and Wei Cheng · 2024
Cited alongside, same era.
Dstc: Direct preference learning with only self-generated tests and code to improve code lms, 2024
Zhihan Liu, Shenao Zhang, Yongfei Liu, Boyi Liu, Yingxiang Yang, and Zhaoran Wang · 2024
Cited alongside, same era.
Nemotron-4 340b technical report, 2024
Nvidia · 2024
Cited alongside, same era.
Code generation with alphacodium: From prompt engineering to flow engineering, 2024
Codejudge-eval: Can large language models be good judges in code understanding?, 2024
Yuwei Zhao, Ziyang Luo, Yuchen Tian, Hongzhan Lin, Weixiang Yan, Annan Li, and Jing Ma · 2024
Later among the works it cites.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025
DeepSeek-AI · 2025
Closest in time.
Dynamic scaling of unit tests for code reward modeling, 2025
Zeyao Ma, Xiaokang Zhang, Jing Zhang, Jifan Yu, Sijia Luo, and Jie Tang · 2025
Closest in time.
Swt-bench: Testing and validating real-world bug-fixes with code agents, 2025
Niels Mündler, Mark Niklas Müller, Jingxuan He, and Martin Vechev · 2025
Closest in time.
Coffe: A code efficiency benchmark for code generation, 2025
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Tal Ridnik, Dedy Kredo, and Itamar Friedman · 2024
Cited alongside, same era.
Gabriel Ryan, Siddhartha Jain, Mingyue Shang, Shiqi Wang, Xiaofei Ma, Murali Krishna Ramanathan, and Baishakhi Ray · 2024
Cited alongside, same era.
Valtest: Automated validation of language model generated test cases, 2024
Hamed Taherkhani and Hadi Hemmati · 2024
Cited alongside, same era.
Selfcodealign: Self-alignment for code generation, 2024
Yuxiang Wei, Federico Cassano, Jiawei Liu, Yifeng Ding, Naman Jain, Zachary Mueller, Harm de Vries, Leandro von Werra, Arjun Guha, and Lingming Zhang · 2024
Cited alongside, same era.
Code optimization chain-of-thought: Structured understanding and self-checking
Qingyao Xu, Dingkang Yang, and Lihua Zhang · 2024
Cited alongside, same era.
Testbench: Evaluating class-level test case generation capability of large language models, 2024
Quanjun Zhang, Ye Shang, Chunrong Fang, Siqi Gu, Jianyi Zhou, and Zhenyu Chen · 2024
Cited alongside, same era.
Testgeneval: A real world unit test generation and test completion benchmark, 2024a
Kush Jain, Gabriel Synnaeve, and Baptiste Rozière
Cited in the paper.
R2E: Turning any github repository into a programming agent environment
Naman Jain, Manish Shetty, Tianjun Zhang, King Han, Koushik Sen, and Ion Stoica
Cited in the paper.
Yun Peng, Jun Wan, Yichen Li, and Xiaoxue Ren · 2025
Closest in time.
Qwen2.5 technical report, 2025
Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu · 2025
Closest in time.
Testeval: Benchmarking large language models for test case generation, 2025
Wenhan Wang, Chenyuan Yang, Zhijie Wang, Yuheng Huang, Zhaoyang Chu, Da Song, Lingming Zhang, An Ran Chen, and Lei Ma · 2025
Closest in time.
Process-supervised reinforcement learning for code generation, 2025
Yufan Ye, Ting Zhang, Wenbin Jiang, and Hua Huang · 2025
Closest in time.
Acecoder: Acing coder rl via automated test-case synthesis, 2025
Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen · 2025
Closest in time.
Generative verifiers: Reward modeling as next-token prediction, 2025
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal · 2025
Closest in time.