Is this Snippet Written by ChatGPT? An Empirical Study with a CodeBERT-Based Classifier

H. Husain, H. Wu, T. Gazit, M. Allamanis, and M. Brockschmidt, “Codesearchnet challenge: Evaluating the state of semantic code search,” CoRR , vol. abs/1909.09436, 2019. [Online]. Available: http://arxiv.org/abs/1909.09436

Original

1909

Earlier work this paper cites.

Z. Li, G. Q. Chen, C. Chen, Y. Zou, and S. Xu, “Ropgen: Towards robust code authorship attribution via automatic coding style transformation,” in Proceedings of the 44th International Conference on Software Engineering , ser. ICSE ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 1906–1918. [Online]. Available: https://doi.org/10.1145/3510003.3510181

1918

Earlier work this paper cites.

Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages,” Psychometrika , vol. 12, no. 2, pp. 153–157, 1947

1947

Earlier work this paper cites.

S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian journal of statistics , pp. 65–70, 1979

1979

Earlier work this paper cites.

Y. D. Liang, Introduction to Java programming . Pearson Education India, 2003

2003

Earlier work this paper cites.

D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. Liu, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu, M. Tufano, S. K. Deng, C. Clement, D. Drain, N. Sundaresan, J. Yin, D. Jiang, and M. Zhou, “Graphcodebert: Pre-training code representations with data flow,” no. arXiv:2009.08366, Sep 2021, arXiv:2009.08366 [cs]. [Online]. Available: http://arxiv.org/abs/2009.08366

Original

2009

Earlier work this paper cites.

T. Menzies, A. Butcher, D. R. Cok, A. Marcus, L. Layman, F. Shull, B. Turhan, and T. Zimmermann, “Local versus global lessons for defect prediction and effort estimation,” IEEE Trans. Software Eng. , vol. 39, no. 6, pp. 822–834, 2013. [Online]. Available: https://doi.org/10.1109/TSE.2012.83

2012

Earlier work this paper cites.

A. Bosu, M. Greiler, and C. Bird, “Characteristics of useful code reviews: An empirical study at microsoft,” in Proceedings of the International Conference on Mining Software Repositories . IEEE - Institute of Electrical and Electronics Engineers, May 2015. [Online]. Available: https://www.microsoft.com/en-us/research/publication/characteristics-of-useful-code-reviews-an-empirical-study-at-microsoft/

2015

Earlier work this paper cites.

H. Dalianis, Evaluation Metrics and Evaluation . Cham: Springer International Publishing, 2018, pp. 45–53. [Online]. Available: https://doi.org/10.1007/978-3-319-78503-5_6

2018

Earlier work this paper cites.

N. Ogura, S. Matsumoto, H. Hata, and S. Kusumoto, “Bring your own coding style,” in 25th International Conference on Software Analysis, Evolution and Reengineering, SANER 2018, Campobasso, Italy, March 20-23, 2018 , R. Oliveto, M. D. Penta, and D. C. Shepherd, Eds. IEEE Computer Society, 2018, pp. 527–531. [Online]. Available: https://doi.org/10.1109/SANER.2018.8330253

2018

Earlier work this paper cites.

T. Dey, S. Mousavi, E. Ponce, T. Fry, B. Vasilescu, A. Filippova, and A. Mockus, “Detecting and characterizing bots that commit code,” in Proceedings of the 17th International Conference on Mining Software Repositories , ser. MSR ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 209–219. [Online]. Available: https://doi.org/10.1145/3379597.3387478

2020

Cited alongside, same era.

Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou, “CodeBERT: A pre-trained model for programming and natural languages,” in Findings of the Association for Computational Linguistics: EMNLP 2020 . Online: Association for Computational Linguistics, Nov. 2020, pp. 1536–1547. [Online]. Available: https://aclanthology.org/2020.findings-emnlp.139

2020

Cited alongside, same era.

R. Morales, F. Khomh, and G. Antoniol, “RePOR: Mimicking humans on refactoring tasks. Are we there yet?” Empirical Software Engineering , vol. 25, no. 4, pp. 2960–2996, Jul. 2020. [Online]. Available: https://doi.org/10.1007/s10664-020-09826-7

2020

Cited alongside, same era.

F. Reda, “GitHub Copilot is not infringing your copyright https://felixreda.eu/2021/07/github-copilot-is-not-infringing-your-copyright/ .”

2021

Later among the works it cites.

Y. Wang, W. Wang, S. Joty, and S. C. Hoi, “Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,” arXiv preprint arXiv:2109.00859 , 2021

Original

2021

Later among the works it cites.

J. Gu, P. Salza, and H. C. Gall, “Assemble Foundation Models for Automatic Code Summarization,” in 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) , Mar. 2022, pp. 935–946, iSSN: 1534-5351. [Online]. Available: 10.1109/SANER53432.2022.00112

2022

Later among the works it cites.

D. Wang, Z. Jia, S. Li, Y. Yu, Y. Xiong, W. Dong, and X. Liao, “Bridging pre-trained models and downstream tasks for source code understanding,” in Proceedings of the 44th International Conference on Software Engineering , ser. ICSE ’22. New York, NY, USA: Association for Computing Machinery, Jul. 2022, pp. 287–298. [Online]. Available: https://dl.acm.org/doi/10.1145/3510003.3510062