An Analysis of the Automatic Bug Fixing Performance of ChatGPT

C. Le Goues, T. Nguyen, S. Forrest, and W. Weimer, “GenProg: A generic method for automatic software repair,” Ieee transactions on software engineering , vol. 38, no. 1, pp. 54–72, 2011

2011

Earlier work this paper cites.

B. Cornu, T. Durieux, L. Seinturier, and M. Monperrus, “NPEfix: Automatic runtime repair of null pointer exceptions in Java,” arXiv preprint arXiv:1512.07423 , 2015

Original

2015

Earlier work this paper cites.

T. Durieux and M. Monperrus, “Dynamoth: dynamic code synthesis for automatic program repair,” in Proceedings of the 11th International Workshop on Automation of Software Test , 2016, pp. 85–91

2016

Earlier work this paper cites.

J. Xuan, M. Martinez, F. Demarco, M. Clement, S. L. Marcote, T. Durieux, D. Le Berre, and M. Monperrus, “Nopol: Automatic repair of conditional statement bugs in Java programs,” IEEE Transactions on Software Engineering , vol. 43, no. 1, pp. 34–55, 2016

2016

Earlier work this paper cites.

W. E. Wong, X. Li, and P. A. Laplante, “Be more familiar with our enemies and pave the way forward: A review of the roles bugs played in software failures,” Journal of Systems and Software , vol. 133, pp. 68–94, 2017

2017

Earlier work this paper cites.

S. O. Haraldsson, J. R. Woodward, A. E. Brownlee, and K. Siggeirsdottir, “Fixing bugs in your sleep: How genetic improvement became an overnight success,” in Proceedings of the Genetic and Evolutionary Computation Conference Companion , 2017, pp. 1513–1520

2017

Earlier work this paper cites.

J. Petke, S. O. Haraldsson, M. Harman, W. B. Langdon, D. R. White, and J. R. Woodward, “Genetic improvement of software: a comprehensive survey,” IEEE Transactions on Evolutionary Computation , vol. 22, no. 3, pp. 415–432, 2017

2017

Earlier work this paper cites.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems , 2017, pp. 5998–6008

2017

Cited alongside, same era.

D. Lin, J. Koppel, A. Chen, and A. Solar-Lezama, “QuixBugs: A multi-lingual program repair benchmark set based on the Quixey Challenge,” in Proceedings Companion of the 2017 ACM SIGPLAN international conference on systems, programming, languages, and applications: software for humanity , 2017, pp. 55–56

2017

Cited alongside, same era.

L. Gazzola, D. Micucci, and L. Mariani, “Automatic software repair: a survey,” in Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018 , M. Chaudron, I. Crnkovic, M. Chechik, and M. Harman, Eds. ACM, 2018, p. 1219. [Online]. Available: https://doi.org/10.1145/3180155.3182526

2018

Cited alongside, same era.

C. Clement, D. Drain, J. Timcheck, A. Svyatkovskiy, and N. Sundaresan, “PyMT5: Multi-mode translation of natural language and Python code with transformers,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2020, pp. 9052–9065

2020

Later among the works it cites.

T. Lutellier, H. V. Pham, L. Pang, Y. Li, M. Wei, and L. Tan, “CoCoNuT: combining context-aware neural translation models using ensemble for program repair,” in ISSTA ’20: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event, USA, July 18-22, 2020 , S. Khurshid and C. S. Pasareanu, Eds. ACM, 2020, pp. 101–114. [Online]. Available: https://doi.org/10.1145/3395363.3397369

2020

Later among the works it cites.

M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al. , “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374 , 2021

Original

2021

Later among the works it cites.