Fetching the paper…
Reading the bibliography…
As large language models (LLMs) become integral to code-related tasks, a central question emerges: Do LLMs truly understand program semantics? We introduce EquiBench, a new benchmark for evaluating LLMs through equivalence checking, i.e., determining whether two programs produce identical outputs for all possible inputs.
Winnowing: local algorithms for document fingerprinting
Saul Schleimer, Daniel S Wilkerson, and Alex Aiken. 2003 · 2003
Earlier work this paper cites.
Llvm: A compilation framework for lifelong program analysis & transformation
Chris Lattner and Vikram Adve. 2004 · 2004
Earlier work this paper cites.
Automatic mining of functionally equivalent code fragments via random testing
Lingxiao Jiang and Zhendong Su. 2009 · 2009
Earlier work this paper cites.
Finding and understanding bugs in c compilers
Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011 · 2011
Earlier work this paper cites.
Stochastic superoptimization
Eric Schkufza, Rahul Sharma, and Alex Aiken. 2013 · 2013
Earlier work this paper cites.
Data-driven equivalence checking
Rahul Sharma, Eric Schkufza, Berkeley Churchill, and Alex Aiken. 2013 · 2013
Earlier work this paper cites.
Automating regression verification
Dennis Felsing, Sarah Grebing, Vladimir Klebanov, Philipp Rümmer, and Mattias Ulbrich. 2014 · 2014
Earlier work this paper cites.
Deep learning code fragments for code clone detection
Martin White, Michele Tufano, Christopher Vendome, and Denys Poshyvanyk. 2016 · 2016
Earlier work this paper cites.
Black-box equivalence checking across compiler optimizations
Manjeet Dahiya and Sorav Bansal. 2017 · 2017
Earlier work this paper cites.
Refdiff: Detecting refactorings in version histories
Danilo Silva and Marco Tulio Valente. 2017 · 2017
Earlier work this paper cites.
Tvm: An automated end-to-end optimizing compiler for deep learning
Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, and 1 others. 2018 · 2018
Earlier work this paper cites.
Think you have solved question answering? try arc, the ai2 reasoning challenge
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018 · 2018
Earlier work this paper cites.
Effective use of smt solvers for program equivalence checking through invariant-sketching and query-decomposition
Shubhani Gupta, Aseem Saxena, Anmol Mahajan, and Sorav Bansal. 2018 · 2018
Earlier work this paper cites.
Client-specific equivalence checking
Federico Mora, Yi Li, Julia Rubin, and Marsha Chechik. 2018 · 2018
Earlier work this paper cites.
Semantic program alignment for equivalence checking
Berkeley Churchill, Oded Padon, Rahul Sharma, and Alex Aiken. 2019 · 2019
Earlier work this paper cites.
Ardiff: scaling program equivalence checking via iterative abstraction and refinement of common code
Sahar Badihi, Faridah Akinotcho, Yi Li, and Julia Rubin. 2020 · 2020
Earlier work this paper cites.
Program synthesis with large language models
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021 · 2021
Earlier work this paper cites.
Eqbench: A dataset of equivalent and non-equivalent program pairs
Sahar Badihi, Yi Li, and Julia Rubin. 2021 · 2021
Earlier work this paper cites.
Evaluating large language models trained on code
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021 · 2021
Earlier work this paper cites.
Training verifiers to solve math word problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021 · 2021
Cited alongside, same era.
Adaptive restarts for stochastic synthesis
Jason R Koenig, Oded Padon, and Alex Aiken. 2021 · 2021
Cited alongside, same era.
CodeXGLUE: A machine learning benchmark dataset for code understanding and generation
Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, MING GONG, Ming Zhou, Nan Duan, Neel Sundaresan, and 3 others. 2021 · 2021
Cited alongside, same era.
Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks
Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, and 1 others. 2021 · 2021
Cited alongside, same era.
Meta large language model compiler: Foundation models of compiler optimization
Chris Cummins, Volker Seeker, Dejan Grubisic, Baptiste Roziere, Jonas Gehring, Gabriel Synnaeve, and Hugh Leather. 2024 · 2024
Later among the works it cites.
Semcoder: Training code language models with comprehensive semantics reasoning
Yangruibo Ding, Jinjun Peng, Marcus J Min, Gail Kaiser, Junfeng Yang, and Baishakhi Ray. 2024 · 2024
Later among the works it cites.
Virtual compiler is all you need for assembly code search
Zeyu Gao, Hao Wang, Yuanda Wang, and Chao Zhang. 2024 · 2024
Later among the works it cites.
Cruxeval: A benchmark for code reasoning, understanding and execution
Alex Gu, Baptiste Rozière, Hugh Leather, Armando Solar-Lezama, Gabriel Synnaeve, and Sida I Wang. 2024 · 2024
Later among the works it cites.
Repository-level compositional code translation and validation
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Lianmin Zheng, Ruochen Liu, Junru Shao, Tianqi Chen, Joseph E Gonzalez, Ion Stoica, and Ameer Haj Ali. 2021 · 2021
Cited alongside, same era.
Large language models are reasoning teachers
Namgyu Ho, Laura Schmid, and Se-Young Yun. 2022 · 2022
Cited alongside, same era.
Towards reasoning in large language models: A survey
Jie Huang and Kevin Chen-Chuan Chang. 2022 · 2022
Cited alongside, same era.
Finding missed optimizations through the lens of dead code elimination
Theodoros Theodoridis, Manuel Rigger, and Zhendong Su. 2022 · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022 · 2022
Cited alongside, same era.
Least-to-most prompting enables complex reasoning in large language models
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and 1 others. 2022 · 2022
Cited alongside, same era.
Sparks of artificial general intelligence: Early experiments with gpt-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, and 1 others. 2023 · 2023
Cited alongside, same era.
Large language models for compiler optimization
Chris Cummins, Volker Seeker, Dejan Grubisic, Mostafa Elhoushi, Youwei Liang, Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Kim Hazelwood, Gabriel Synnaeve, and 1 others. 2023 · 2023
Cited alongside, same era.
Ali Reza Ibrahimzada, Kaiyao Ke, Mrigank Pawagi, Muhammad Salman Abid, Rangeet Pan, Saurabh Sinha, and Reyhaneh Jabbarvand. 2024 · 2024
Later among the works it cites.
Livecodebench: Holistic and contamination free evaluation of large language models for code
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024 · 2024
Later among the works it cites.
Code simulation challenges for large language models
Emanuele La Malfa, Christoph Weinhuber, Orazio Torre, Fangru Lin, Samuele Marro, Anthony Cohn, Nigel Shadbolt, and Michael Wooldridge. 2024 · 2024
Later among the works it cites.
Codemind: A framework to challenge large language models for code reasoning
Changshu Liu, Shizhuo Dylan Zhang, Ali Reza Ibrahimzada, and Reyhaneh Jabbarvand. 2024 · 2024
Later among the works it cites.
What can large language models capture about code functional equivalence?
Nickil Maveli, Antonio Vergari, and Shay B Cohen. 2024 · 2024
Later among the works it cites.
Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models
Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2024 · 2024
Later among the works it cites.
Next: Teaching large language models to reason about code execution
Ansong Ni, Miltiadis Allamanis, Arman Cohan, Yinlin Deng, Kensen Shi, Charles Sutton, and Pengcheng Yin. 2024 · 2024
Later among the works it cites.
Semantic code refactoring for abstract data types
Shankara Pailoor, Yuepeng Wang, and Işıl Dillig. 2024 · 2024
Later among the works it cites.
Lost in translation: A study of bugs introduced by large language models while translating code
Rangeet Pan, Ali Reza Ibrahimzada, Rahul Krishna, Divya Sankar, Lambert Pouguem Wassi, Michele Merler, Boris Sobolev, Raju Pavuluri, Saurabh Sinha, and Reyhaneh Jabbarvand. 2024 · 2024
Later among the works it cites.
Exploring the use of llms for sql equivalence checking
Rajat Singh and Srikanta Bedathur. 2024 · 2024
Later among the works it cites.
Large language models for equivalent mutant detection: How far are we?
Zhao Tian, Honglin Shu, Dong Wang, Xuejie Cao, Yasutaka Kamei, and Junjie Chen. 2024 · 2024
Later among the works it cites.
Transformer-based models are not yet perfect at learning to emulate structural recursion
Dylan Zhang, Curt Tigges, Zory Zhang, Stella Biderman, Maxim Raginsky, and Talia Ringer. 2024 · 2024
Later among the works it cites.
Dce-llm: Dead code elimination with large language models
Minyu Chen, Guoqiang Li, Ling-I Wu, and Ruibang Liu. 2025 · 2025
Closest in time.
Python variable renaming tool
Daniel Flook. 2025 · 2025
Closest in time.