From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future · Around

Fetching the paper…

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future · Around

Built on

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation
1997
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation
1997
Earlier work this paper cites.
MIT press, 1999
C. Manning and H. Schutze, Foundations of statistical natural language processing · 1999
Earlier work this paper cites.
N. R. Jennings, “A survey of agent-oriented software engineering,” Knowledge Engineering Review
2000
Earlier work this paper cites.
C. Flanagan and K. R. M. Leino, “Houdini, an annotation assistant for esc/java,” in FME 2001: Formal Methods for Increasing Software Productivity
2001
Earlier work this paper cites.
S. Wang, D. Chollak, D. Movshovitz-Attias, and L. Tan, “Bugram: bug detection with n-gram language models,” in Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering
2016
Earlier work this paper cites.
Pearson Education Limited, 2016
S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach · 2016
Earlier work this paper cites.
J. X. Chen, “The evolution of computing: Alphago,” Computing in Science & Engineering
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems
2017
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805
Original
2018
Earlier work this paper cites.
A. Vogelsang and M. Borg, “Requirements engineering for machine learning: Perspectives from data scientists,” in 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW)
2019
Earlier work this paper cites.
P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” in Advances in Neural Information Processing Systems
2020
Earlier work this paper cites.
Y. Bisk, A. Holtzman, J. Thomason, J. Andreas, Y. Bengio, J. Chai, M. Lapata, A. Lazaridou, J. May, A. Nisnevich, N. Pinto, and J. Turian, “Experience grounds language,” 2020
2020
Earlier work this paper cites.
L. Floridi and M. Chiriatti, “Gpt-3: Its nature, scope, limits, and consequences,” Minds and Machines
2020
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al
2020
Earlier work this paper cites.
T. Hey, J. Keim, A. Koziolek, and W. F. Tichy, “Norbert: Transfer learning for requirements classification,” in 2020 IEEE 28th International Requirements Engineering Conference (RE)
2020
Earlier work this paper cites.
M. Tufano, D. Drain, A. Svyatkovskiy, S. K. Deng, and N. Sundaresan, “Unit test case generation with transformers and focal context,” arXiv preprint arXiv:2009.05617
Original
2020
Earlier work this paper cites.
arXiv:2107.03374 [cs.LG]
Original
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374 · 2021
Earlier work this paper cites.
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating large language models trained on code,” 2021
2021
Earlier work this paper cites.
Y. Wang, W. Wang, S. Joty, and S. C. Hoi, “Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,” arXiv preprint arXiv:2109.00859
Original
2021
Earlier work this paper cites.
N. Jain, S. Vaidyanath, A. Iyer, N. Natarajan, S. Parthasarathy, S. Rajamani, and R. Sharma, “Jigsaw: large language models meet program synthesis,” in Proceedings of the 44th International Conference on Software Engineering
2022
Earlier work this paper cites.
J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al
2022
Earlier work this paper cites.
C. Thapa, S. I. Jang, M. E. Ahmed, S. Camtepe, J. Pieprzyk, and S. Nepal, “Transformer-based language models for software vulnerability detection,” in Proceedings of the 38th Annual Computer Security Applications Conference
2022
Earlier work this paper cites.
S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, T. Mihaylov, M. Ott, S. Shleifer, K. Shuster, D. Simig, P. S. Koura, A. Sridhar, T. Wang, and L. Zettlemoyer, “Opt: Open pre-trained transformer language models,” 2022
2022
Earlier work this paper cites.
S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629
Original
2022
Earlier work this paper cites.
W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” in International conference on machine learning
2022
Earlier work this paper cites.
X. Luo, Y. Xue, Z. Xing, and J. Sun, “Prcbert: Prompt learning for requirement classification using bert-based pretrained language models,” in Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering
2022
Earlier work this paper cites.
J. Huang, S. S. Gu, L. Hou, Y. Wu, X. Wang, H. Yu, and J. Han, “Large language models can self-improve,” arXiv preprint arXiv:2210.11610
Original
2022
Earlier work this paper cites.
H. J. Kang, T. G. Nguyen, B. Le, C. S. Păsăreanu, and D. Lo, “Test mimicry to assess the exploitability of library vulnerabilities,” in Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis
2022
Earlier work this paper cites.
A. Fan, B. Gokkaya, M. Harman, M. Lyubarskiy, S. Sengupta, S. Yoo, and J. M. Zhang, “Large language models for software engineering: Survey and open problems,” in 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE)
2023
Earlier work this paper cites.
Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou, R. Zheng, X. Fan, X. Wang, L. Xiong, Y. Zhou, W. Wang, C. Jiang, Y. Zou, X. Liu, Z. Yin, S. Dou, R. Weng, W. Cheng, Q. Zhang, W. Qin, Y. Zheng, X. Qiu, X. Huang, and T. Gui, “The rise and potential of large language model based agents: A survey,” 2023
2023
Earlier work this paper cites.
Z. Zheng, K. Ning, J. Chen, Y. Wang, W. Chen, L. Guo, and W. Wang, “Towards an understanding of large language models in software engineering tasks,” 2023
2023
Earlier work this paper cites.
A. Nguyen-Duc, B. Cabrero-Daniel, A. Przybylek, C. Arora, D. Khanna, T. Herda, U. Rafiq, J. Melegati, E. Guerra, K.-K. Kemell, M. Saari, Z. Zhang, H. Le, T. Quan, and P. Abrahamsson, “Generative artificial intelligence for software engineering – a research agenda,” 2023
2023
Earlier work this paper cites.
Z. Rasheed, M. Waseem, K.-K. Kemell, W. Xiaofeng, A. N. Duc, K. Systä, and P. Abrahamsson, “Autonomous agents in software development: A vision paper,” arXiv preprint arXiv:2311.18440
Original
2023
Earlier work this paper cites.
A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al
2023
Earlier work this paper cites.
Y. Wang, H. Le, A. D. Gotmare, N. D. Bui, J. Li, and S. C. Hoi, “Codet5+: Open code large language models for code understanding and generation,” arXiv preprint arXiv:2305.07922
Original
2023
Earlier work this paper cites.
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al
Original
2023
Earlier work this paper cites.
G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, “Voyager: An open-ended embodied agent with large language models,” arXiv preprint arXiv:2305.16291
Original
2023
Earlier work this paper cites.
C. Whitehouse, M. Choudhury, and A. F. Aji, “Llm-powered data augmentation for enhanced cross-lingual performance,” 2023
2023
Earlier work this paper cites.
J. White, Q. Fu, S. Hays, M. Sandborn, C. Olea, H. Gilbert, A. Elnashar, J. Spencer-Smith, and D. C. Schmidt, “A prompt pattern catalog to enhance prompt engineering with chatgpt,” arXiv preprint arXiv:2302.11382
Original
2023
Earlier work this paper cites.
Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” ACM Computing Surveys
2023
Earlier work this paper cites.
S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, C. Zhang, J. Wang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, C. Ran, L. Xiao, C. Wu, and J. Schmidhuber, “Metagpt: Meta programming for a multi-agent collaborative framework,” 2023
2023
Earlier work this paper cites.
W. Chen, Y. Su, J. Zuo, C. Yang, C. Yuan, C. Qian, C.-M. Chan, Y. Qin, Y. Lu, R. Xie, et al
Original
2023
Earlier work this paper cites.
J. Zhang, Y. Chen, N. Niu, and C. Liu, “Evaluation of chatgpt on requirements information retrieval under zero-shot setting,” Available at SSRN 4450322
2023
Earlier work this paper cites.
A. Moharil and A. Sharma, “Identification of intra-domain ambiguity using transformer-based machine learning,” in Proceedings of the 1st International Workshop on Natural Language-Based Software Engineering
2023
Earlier work this paper cites.
A. Poudel, J. Lin, and J. Cleland-Huang, “Leveraging transformer-based language models to automate requirements satisfaction assessment,” 2023
2023
Earlier work this paper cites.
K. Ronanki, C. Berger, and J. Horkoff, “Investigating chatgpt’s potential to assist in requirements elicitation processes,” in 2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)
2023
Earlier work this paper cites.
D. Xie, B. Yoo, N. Jiang, M. Kim, L. Tan, X. Zhang, and J. S. Lee, “Impact of large language models on generating software specifications,” 2023
2023
Earlier work this paper cites.
A. Moharil and A. Sharma, “Tabasco: A transformer based contextualization toolkit,” Science of Computer Programming
2023
Earlier work this paper cites.
A. Ni, P. Yin, Y. Zhao, M. Riddell, T. Feng, R. Shen, S. Yin, Y. Liu, S. Yavuz, C. Xiong, S. Joty, Y. Zhou, D. Radev, and A. Cohan, “L2ceval: Evaluating language-to-code generation capabilities of large language models,” 2023
2023
Earlier work this paper cites.
R. Sun, S. Ö. Arik, A. Muzio, L. Miculicich, S. Gundabathula, P. Yin, H. Dai, H. Nakhost, R. Sinha, Z. Wang, et al
Original
2023
Earlier work this paper cites.
S. Peng, E. Kalliamvakou, P. Cihon, and M. Demirer, “The impact of ai on developer productivity: Evidence from github copilot,” 2023
2023
Earlier work this paper cites.
D. Fried, A. Aghajanyan, J. Lin, S. Wang, E. Wallace, F. Shi, R. Zhong, W. tau Yih, L. Zettlemoyer, and M. Lewis, “Incoder: A generative model for code infilling and synthesis,” 2023
2023
Earlier work this paper cites.
E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong, “Codegen: An open large language model for code with multi-turn program synthesis,” 2023
2023
Earlier work this paper cites.
K. Zhang, D. Wang, J. Xia, W. Y. Wang, and L. Li, “Algo: Synthesizing algorithmic programs with llm-generated oracle verifiers,” 2023
2023
Earlier work this paper cites.
D. Huang, Q. Bu, J. M. Zhang, M. Luck, and H. Cui, “Agentcoder: Multi-agent-based code generation with iterative testing and optimisation,” arXiv preprint arXiv:2312.13010
Original
2023
Earlier work this paper cites.
C. Qian, X. Cong, C. Yang, W. Chen, Y. Su, J. Xu, Z. Liu, and M. Sun, “Communicative agents for software development,” arXiv preprint arXiv:2307.07924
Original
2023
Earlier work this paper cites.
Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, et al
Original
2023
Earlier work this paper cites.
Y. Li, Y. Zhang, and L. Sun, “Metaagents: Simulating interactions of human behaviors for llm-based task-oriented coordination via collaborative generative agents,” arXiv preprint arXiv:2310.06500
Original
2023
Earlier work this paper cites.
V. Murali, C. Maddila, I. Ahmad, M. Bolin, D. Cheng, N. Ghorbani, R. Fernandez, and N. Nagappan, “Codecompose: A large-scale industrial deployment of ai-assisted code authoring,” arXiv preprint arXiv:2305.12050
Original
2023
Earlier work this paper cites.
X. Chen, M. Lin, N. Schärli, and D. Zhou, “Teaching large language models to self-debug,” arXiv preprint arXiv:2304.05128
Original
2023
Earlier work this paper cites.
S. Kang, B. Chen, S. Yoo, and J.-G. Lou, “Explainable automated debugging via large language model-driven scientific debugging,” arXiv preprint arXiv:2304.02195
Original
2023
Earlier work this paper cites.
G. Franceschelli and M. Musolesi, “On the creativity of large language models,” arXiv preprint arXiv:2304.00008
Original
2023
Earlier work this paper cites.
J. Lai, W. Gan, J. Wu, Z. Qi, and P. S. Yu, “Large language models in law: A survey,” 2023
2023
Earlier work this paper cites.
L. Chen, Y. Zhang, S. Ren, H. Zhao, Z. Cai, Y. Wang, P. Wang, T. Liu, and B. Chang, “Towards end-to-end embodied decision making via multi-modal large language model: Explorations with gpt4-vision and beyond,” arXiv preprint arXiv:2310.02071
Original
2023
Earlier work this paper cites.
G. Li, H. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem, “Camel: Communicative agents for” mind” exploration of large language model society,” Advances in Neural Information Processing Systems
2023
Earlier work this paper cites.
Z. Liu, W. Yao, J. Zhang, L. Xue, S. Heinecke, R. Murthy, Y. Feng, Z. Chen, J. C. Niebles, D. Arpit, et al
Original
2023
Earlier work this paper cites.

Similar

R. Feldt, S. Kang, J. Yoon, and S. Yoo, “Towards autonomous testing agents via conversational large language models,” in 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)
2023
Cited alongside, same era.
A. Happe and J. Cito, “Getting pwn’d by ai: Penetration testing with large language models,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
2023
Cited alongside, same era.
G. Sridhara, S. Mazumdar, et al
Original
2023
Cited alongside, same era.
K. Kolthoff, C. Bartelt, and S. P. Ponzetto, “Data-driven prototyping via natural-language-based gui retrieval,” Automated software engineering
2023
Cited alongside, same era.
S. Jalil, S. Rafi, T. D. LaToza, K. Moran, and W. Lam, “Chatgpt and software testing education: promises & perils (2023),” arXiv preprint arXiv:2302.03287

Then

Y. Dong, X. Jiang, Z. Jin, and G. Li, “Self-collaboration code generation via chatgpt,” 2024
2024
Closest in time.
F. Lin, D. J. Kim, et al
Original
2024
Closest in time.
M. A. Islam, M. E. Ali, and M. R. Parvez, “Mapcoder: Multi-agent code generation for competitive problem solving,” arXiv preprint arXiv:2405.11403
Original
2024
Closest in time.
S. Holt, M. R. Luyten, and M. van der Schaar, “L2MAC: Large language model automatic computer for extensive code generation,” in The Twelfth International Conference on Learning Representations
2024
Closest in time.
Y. Ishibashi and Y. Nishimura, “Self-organized agents: A llm multi-agent framework toward ultra large-scale code generation and optimization,” 2024
2024

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…