Fetching the paper…
Reading the bibliography…
Various techniques have been proposed to leverage the capabilities of code language models (CLMs) for SE tasks.
J. Bieri, “Cognitive complexity-simplicity and predictive behavior.” The Journal of Abnormal and Social Psychology , vol. 51, no. 2, p. 263, 1955
1955
Earlier work this paper cites.
F. Jelinek, R. L. Mercer, L. R. Bahl, and J. K. Baker, “Perplexity—a measure of the difficulty of speech recognition tasks,” The Journal of the Acoustical Society of America , vol. 62, no. S1, pp. S63–S63, 1977
1977
Earlier work this paper cites.
P. Oman and J. Hagemeister, “Metrics for assessing a software system’s maintainability,” in Proceedings Conference on Software Maintenance 1992 , 1992, pp. 337–344
1992
Earlier work this paper cites.
D. Coleman, D. Ash, B. Lowther, and P. Oman, “Using metrics to evaluate software system maintainability,” Computer , vol. 27, no. 8, pp. 44–49, 1994
1994
Earlier work this paper cites.
A. Z. Broder, “Identifying and filtering near-duplicate documents,” in Combinatorial Pattern Matching , R. Giancarlo and D. Sankoff, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2000, pp. 1–10
2000
Earlier work this paper cites.
J.-l. Gailly and M. Adler, “Zlib compression library,” 2004
2004
Earlier work this paper cites.
M. Kim, L. Bergman, T. Lau, and D. Notkin, “An ethnographic study of copy and paste programming practices in oopl,” in Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE ’04. , 2004, pp. 83–92
2004
Earlier work this paper cites.
2009
Earlier work this paper cites.
https://radon.readthedocs.io/en/latest/intro.html#cyclomatic-complexity , 2012
2012
Earlier work this paper cites.
V. Raychev, M. Vechev, and E. Yahav, “Code completion with statistical language models,” in Proceedings of the 35th ACM SIGPLAN conference on programming language design and implementation , 2014, pp. 419–428
2014
Earlier work this paper cites.
J. Al Dallal and A. Abdin, “Empirical evaluation of the impact of object-oriented code refactoring on quality attributes: A systematic literature review,” IEEE Transactions on Software Engineering , vol. 44, no. 1, pp. 44–69, 2017
2017
Earlier work this paper cites.
S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in 2018 IEEE 31st Computer Security Foundations Symposium (CSF) , 2018, pp. 268–282
2018
Earlier work this paper cites.
N. Carlini, C. Liu, Ú. Erlingsson, J. Kos, and D. Song, “The secret sharer: Evaluating and testing unintended memorization in neural networks,” in 28th USENIX Security Symposium (USENIX Security 19) , 2019, pp. 267–284
2019
Earlier work this paper cites.
A. Svyatkovskiy, Y. Zhao, S. Fu, and N. Sundaresan, “Pythia: Ai-assisted code completion system,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , 2019, pp. 2727–2735
2019
Earlier work this paper cites.
A. A. B. Baqais and M. Alshayeb, “Automatic software refactoring: a systematic literature review,” Software Quality Journal , vol. 28, no. 2, pp. 459–502, 2020
2020
Earlier work this paper cites.
A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, “The curious case of neural text degeneration,” in 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net, 2020. [Online]. Available: https://openreview.net/forum?id=rygGQyrFvH
2020
Earlier work this paper cites.
2020
Earlier work this paper cites.
N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. B. Brown, D. Song, Ú. Erlingsson, A. Oprea, and C. Raffel, “Extracting training data from large language models,” in 30th USENIX Security Symposium, USENIX Security 2021, August 11-13, 2021 , M. D. Bailey and R. Greenstadt, Eds. USENIX Association, 2021, pp. 2633–2650. [Online]. Available: https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting
2021
Earlier work this paper cites.
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating large language models trained on code,” 2021
2021
Earlier work this paper cites.
2021
Earlier work this paper cites.
2021
Earlier work this paper cites.
“openai/human-eval,” https://github.com/openai/human-eval/commits/master/data , 2021
2021
Earlier work this paper cites.
2021
Earlier work this paper cites.
K. Tirumala, A. Markosyan, L. Zettlemoyer, and A. Aghajanyan, “Memorization without overfitting: Analyzing the training dynamics of large language models,” Advances in Neural Information Processing Systems , vol. 35, pp. 38 274–38 290, 2022
2022
Earlier work this paper cites.
N. Kandpal, E. Wallace, and C. Raffel, “Deduplicating training data mitigates privacy risks in language models,” in International Conference on Machine Learning . PMLR, 2022, pp. 10 697–10 707
2022
Earlier work this paper cites.
https://pypi.org/project/cognitive-complexity/ , 2022
2022
Earlier work this paper cites.
D. Kocetkov, R. Li, L. Ben Allal, J. Li, C. Mou, C. Muñoz Ferrandis, Y. Jernite, M. Mitchell, S. Hughes, T. Wolf, D. Bahdanau, L. von Werra, and H. de Vries, “The stack: 3 tb of permissively licensed source code,” Preprint , 2022
2022
Earlier work this paper cites.
Z. Du, Y. Qian, X. Liu, M. Ding, J. Qiu, Z. Yang, and J. Tang, “Glm: General language model pretraining with autoregressive blank infilling,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2022, pp. 320–335
2022
Earlier work this paper cites.
2022
Earlier work this paper cites.
“Github copilot is generally available to all developers,” https://github.blog/2022-06-21-github-copilot-is-generally-available-to- all-developers/ , 2022
2022
Cited alongside, same era.
2022
Cited alongside, same era.
Y. Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang, “Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,” in Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis , 2023, pp. 423–435
2023
Cited alongside, same era.
C. S. Xia, Y. Wei, and L. Zhang, “Automated program repair in the era of large pre-trained language models,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) . IEEE, 2023, pp. 1482–1494
2023
Cited alongside, same era.
2023
Later among the works it cites.
“Wizardlm/wizardcoder,” https://huggingface.co/WizardLM/WizardCoder-15B-V1.0 , 2023
2023
Later among the works it cites.
S. Chaudhary, “Code alpaca: An instruction-following llama model for code generation,” https://github.com/sahil280114/codealpaca , 2023
2023
Later among the works it cites.
“sahil280114/codealpaca,” https://github.com/sahil280114/codealpaca/commit/d269da106a579a623a654529b3cb91b5dfa9c72f , 2023
2023
Later among the works it cites.
“codellama/codellama-7b-instruct-hf,” https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf , 2023
2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2023
Cited alongside, same era.
O. Sainz, J. Campos, I. García-Ferrero, J. Etxaniz, O. L. de Lacalle, and E. Agirre, “NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 10 776–10 787. [Online]. Available: https://aclanthology.org/2023.findings-emnlp.722
2023
Cited alongside, same era.
2023
Cited alongside, same era.
2023
Cited alongside, same era.
Y. Wu, N. Jiang, H. V. Pham, T. Lutellier, J. Davis, L. Tan, P. Babkin, and S. Shah, “How effective are neural networks for fixing security vulnerabilities,” in Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis , ser. ISSTA 2023. New York, NY, USA: Association for Computing Machinery, 2023, p. 1282–1294. [Online]. Available: https://doi.org/10.1145/3597926.3598135
2023
Cited alongside, same era.
T.-O. Li, W. Zong, Y. Wang, H. Tian, Y. Wang, S.-C. Cheung, and J. Kramer, “Nuances are the key: Unlocking chatgpt to find failure-inducing tests with differential prompting,” in 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 2023, pp. 14–26
2023
Cited alongside, same era.
2023
Cited alongside, same era.
2023
Cited alongside, same era.
“codellama/codellama-7b-instruct-hf commit,” https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf/commit/65db8fcae13921e49c3dd0a2be4757102d0e723f , 2023
2023
Later among the works it cites.
“meta-llama/llama-2-7b,” https://huggingface.co/meta-llama/Llama-2-7b , 2023
2023
Later among the works it cites.
“Phind/phind-codellama-34b-v2,” https://huggingface.co/Phind/Phind-CodeLlama-34B-v2 , 2023
2023
Later among the works it cites.
“Phind/phind-codellama-34b-v2,” https://huggingface.co/Phind/Phind-CodeLlama-34B-v2/commit/29c3be6006297754f344ba05678c038b0b77f6c0 , 2023
2023
Later among the works it cites.
“Thudm/chatglm3-6b,” https://huggingface.co/THUDM/chatglm3-6b , 2023
2023
Later among the works it cites.
“Thudm/chatglm3-6b,” https://huggingface.co/THUDM/chatglm3-6b/commit/62acdaa77a5742d120acf9b6419656b403218c3d , 2023
2023
Later among the works it cites.
“Gpt-3.5-turbo model availability,” https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#gpt-35-turbo-model-availability , 2023
2023
Later among the works it cites.
“Github copilot,” https://copilot.microsoft.com/ , 2023
2023
Later among the works it cites.
“Introducing github copilot: Ai pair programmer,” https://github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/ , 2023
2023
Later among the works it cites.
“Github copilot november 30th update,” https://github.blog/changelog/2023-11-30-github-copilot-november-30th-update/ , 2023
2023
Later among the works it cites.
“Gpt-4 and gpt-4-turbo preview,” https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#gpt-4-and-gpt-4-turbo-preview , 2023
2023
Later among the works it cites.
2023
Later among the works it cites.
2023
Later among the works it cites.
J. Liu, C. S. Xia, Y. Wang, and L. Zhang, “Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/forum?id=1qvx610Cu7
2023
Later among the works it cites.
2023
Later among the works it cites.
2023
Later among the works it cites.
Y. Deng, C. S. Xia, C. Yang, S. D. Zhang, S. Yang, and L. Zhang, “Large language models are edge-case generators: Crafting unusual programs for fuzzing deep learning libraries,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering , 2024, pp. 1–13
2024
Closest in time.
C. S. Xia, M. Paltenghi, J. Le Tian, M. Pradel, and L. Zhang, “Fuzz4all: Universal fuzzing with large language models,” Proc. IEEE/ACM ICSE , 2024
2024
Closest in time.
C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. R. Narasimhan, “SWE-bench: Can language models resolve real-world github issues?” in The Twelfth International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=VTF8yNQM66
2024
Closest in time.
N. Gruver, M. Finzi, S. Qiu, and A. G. Wilson, “Large language models are zero-shot time series forecasters,” Advances in Neural Information Processing Systems , vol. 36, 2024
2024
Closest in time.
2024
Closest in time.
https://github.com/johnbumgarner/wordhoard , 2024
2024
Closest in time.
S. Balloccu, P. Schmidtová, M. Lango, and O. Dusek, “Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source LLMs,” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , Y. Graham and M. Purver, Eds. St. Julian’s, Malta: Association for Computational Linguistics, Mar. 2024, pp. 67–93. [Online]. Available: https://aclanthology.org/2024.eacl-long.5
2024
Closest in time.