Built on

D. Spencer, Card sorting: Designing usable categories . Rosenfeld Media, 2009
2009
Earlier work this paper cites.
J. Zhou, H. Zhang, and D. Lo, “Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports,” in 2012 34th International Conference on Software Engineering (ICSE) , 2012, pp. 14–24
2012
Earlier work this paper cites.
J. Zhou, H. Zhang, and D. Lo, “Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports,” in 2012 34th International conference on software engineering (ICSE) . IEEE, 2012, pp. 14–24
2012
Earlier work this paper cites.
M. L. McHugh, “Interrater reliability: the kappa statistic,” Biochemia medica , vol. 22, no. 3, pp. 276–282, 2012
2012
Earlier work this paper cites.
C. Tantithamthavorn, A. Ihara, and K.-i. Matsumoto, “Using co-change histories to improve bug localization performance,” in 2013 14th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing . IEEE, 2013, pp. 543–548
2013
Earlier work this paper cites.
K. Charmaz, “Constructing grounded theory,” 2014
2014
Earlier work this paper cites.
C. Tantithamthavorn, S. L. Abebe, A. E. Hassan, A. Ihara, and K. Matsumoto, “The impact of IR-based classifier configuration on the performance and the effort of method-level bug localization,” Information and Software Technology , vol. 102, pp. 160–174, 2018
2018
Earlier work this paper cites.
T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with bert,” arXiv preprint arXiv:1904.09675 , 2019
Original
2019
Earlier work this paper cites.
J. Pasuksmit, P. Thongtanunam, and S. Karunasekera, “Towards just-enough documentation for agile effort estimation: What information should be documented?” in 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 2021, pp. 114–125
2021
Earlier work this paper cites.
D. Hendrycks, S. Basart, S. Kadavath, M. Mazeika, A. Arora, E. Guo, C. Burns, S. Puranik, H. He, D. Song et al. , “Measuring coding challenge competence with apps,” arXiv preprint arXiv:2105.09938 , 2021
Original
2021
Earlier work this paper cites.
J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le et al. , “Program synthesis with large language models,” arXiv preprint arXiv:2108.07732 , 2021
Original
2021
Earlier work this paper cites.
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al. , “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374 , 2021
Original
2021
Earlier work this paper cites.
D. Zan, B. Chen, F. Zhang, D. Lu, B. Wu, B. Guan, Y. Wang, and J.-G. Lou, “Large language models meet nl2code: A survey,” arXiv preprint arXiv:2212.09420 , 2022
Original
2022
Earlier work this paper cites.
Z. Li, S. Lu, D. Guo, N. Duan, S. Jannu, G. Jenks, D. Majumder, J. Green, A. Svyatkovskiy, S. Fu et al. , “CodeReviewer: Pre-training for automating code review activities,” arXiv preprint arXiv:2203.09095 , 2022
Original
2022
Earlier work this paper cites.
P. Thongtanunam, C. Pornprasit, and C. Tantithamthavorn, “Autotransform: Automated code transformation to support modern code review process,” in Proceedings of the 44th international conference on software engineering , 2022, pp. 237–248
2022
Earlier work this paper cites.

Similar

Y. Hong, C. Tantithamthavorn, P. Thongtanunam, and A. Aleti, “Commentfinder: a simpler, faster, more accurate code review comments recommendation,” in Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software engineering , 2022, pp. 507–519
2022
Cited alongside, same era.
M. Fu and C. Tantithamthavorn, “Linevul: A transformer-based line-level vulnerability prediction,” in Proceedings of the 19th International Conference on Mining Software Repositories , 2022, pp. 608–620
2022
Cited alongside, same era.
M. Fu, C. Tantithamthavorn, T. Le, V. Nguyen, and D. Phung, “Vulrepair: a t5-based automated software vulnerability repair,” in Proceedings of the 30th ACM joint european software engineering conference and symposium on the foundations of software engineering , 2022, pp. 935–947
2022
Cited alongside, same era.
W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223 , 2023

Then

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Human-In-the-Loop Software Development Agents

Takerngsaksiri, Wannita, Pasuksmit, Jirat, Thongtanunam, Patanamon et al.

Understand

Recently, Large Language Models (LLMs)-based multi-agent paradigms for software engineering are introduced to automatically resolve software development tasks (e.g., from a given issue to source code).

However, existing work is evaluated based on historical benchmark datasets, rarely considers human feedback at each stage of the automated software development process, and has not been deployed in practice.
In this paper, we introduce a Human-in-the-loop LLM-based Agents framework (HULA) for software development that allows software engineers to refine and guide LLMs when generating coding plans and source code for a given task.
We design, implement, and deploy the HULA framework into Atlassian JIRA for internal uses.

Built on

D. Spencer, Card sorting: Designing usable categories . Rosenfeld Media, 2009
2009
Earlier work this paper cites.
J. Zhou, H. Zhang, and D. Lo, “Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports,” in 2012 34th International Conference on Software Engineering (ICSE) , 2012, pp. 14–24
2012
Earlier work this paper cites.
J. Zhou, H. Zhang, and D. Lo, “Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports,” in 2012 34th International conference on software engineering (ICSE) . IEEE, 2012, pp. 14–24
2012
Earlier work this paper cites.
M. L. McHugh, “Interrater reliability: the kappa statistic,” Biochemia medica , vol. 22, no. 3, pp. 276–282, 2012
2012
Earlier work this paper cites.
C. Tantithamthavorn, A. Ihara, and K.-i. Matsumoto, “Using co-change histories to improve bug localization performance,” in 2013 14th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing . IEEE, 2013, pp. 543–548
2013
Earlier work this paper cites.
K. Charmaz, “Constructing grounded theory,” 2014
2014
Earlier work this paper cites.
C. Tantithamthavorn, S. L. Abebe, A. E. Hassan, A. Ihara, and K. Matsumoto, “The impact of IR-based classifier configuration on the performance and the effort of method-level bug localization,” Information and Software Technology , vol. 102, pp. 160–174, 2018
2018
Earlier work this paper cites.
T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with bert,” arXiv preprint arXiv:1904.09675 , 2019
Original
2019
Earlier work this paper cites.
J. Pasuksmit, P. Thongtanunam, and S. Karunasekera, “Towards just-enough documentation for agile effort estimation: What information should be documented?” in 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 2021, pp. 114–125
2021
Earlier work this paper cites.
D. Hendrycks, S. Basart, S. Kadavath, M. Mazeika, A. Arora, E. Guo, C. Burns, S. Puranik, H. He, D. Song et al. , “Measuring coding challenge competence with apps,” arXiv preprint arXiv:2105.09938 , 2021
Original
2021
Earlier work this paper cites.
J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le et al. , “Program synthesis with large language models,” arXiv preprint arXiv:2108.07732 , 2021
Original
2021
Earlier work this paper cites.
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al. , “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374 , 2021
Original
2021
Earlier work this paper cites.
D. Zan, B. Chen, F. Zhang, D. Lu, B. Wu, B. Guan, Y. Wang, and J.-G. Lou, “Large language models meet nl2code: A survey,” arXiv preprint arXiv:2212.09420 , 2022
Original
2022
Earlier work this paper cites.
Z. Li, S. Lu, D. Guo, N. Duan, S. Jannu, G. Jenks, D. Majumder, J. Green, A. Svyatkovskiy, S. Fu et al. , “CodeReviewer: Pre-training for automating code review activities,” arXiv preprint arXiv:2203.09095 , 2022
Original
2022
Earlier work this paper cites.
P. Thongtanunam, C. Pornprasit, and C. Tantithamthavorn, “Autotransform: Automated code transformation to support modern code review process,” in Proceedings of the 44th international conference on software engineering , 2022, pp. 237–248
2022
Earlier work this paper cites.

Similar

Y. Hong, C. Tantithamthavorn, P. Thongtanunam, and A. Aleti, “Commentfinder: a simpler, faster, more accurate code review comments recommendation,” in Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software engineering , 2022, pp. 507–519
2022
Cited alongside, same era.
M. Fu and C. Tantithamthavorn, “Linevul: A transformer-based line-level vulnerability prediction,” in Proceedings of the 19th International Conference on Mining Software Repositories , 2022, pp. 608–620
2022
Cited alongside, same era.
M. Fu, C. Tantithamthavorn, T. Le, V. Nguyen, and D. Phung, “Vulrepair: a t5-based automated software vulnerability repair,” in Proceedings of the 30th ACM joint european software engineering conference and symposium on the foundations of software engineering , 2022, pp. 935–947
2022
Cited alongside, same era.
W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223 , 2023

Then

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

2023

Cited alongside, same era.

X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large language models for software engineering: A systematic literature review,” arXiv preprint arXiv:2308.10620 , 2023

Original

2023

Cited alongside, same era.

D. Lo, “Trustworthy and synergistic artificial intelligence for software engineering: Vision and roadmaps,” in 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) , 2023, pp. 69–85

2023

Cited alongside, same era.

R. Hoda, H. Dam, C. Tantithamthavorn, P. Thongtanunam, and M.-A. Storey, “Augmented agile: Human-centered ai-assisted software management,” IEEE Software , vol. 40, no. 4, pp. 106–109, 2023

2023

Cited alongside, same era.

C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan, “Swe-bench: Can language models resolve real-world github issues?” arXiv preprint arXiv:2310.06770 , 2023

Original

2023

Cited alongside, same era.

S. Zhou, U. Alon, S. Agarwal, and G. Neubig, “Codebertscore: Evaluating code generation with pretrained models of code,” arXiv preprint arXiv:2302.05527 , 2023

Original

2023

Cited alongside, same era.

L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing et al. , “Judging llm-as-a-judge with mt-bench and chatbot arena,” Advances in Neural Information Processing Systems , vol. 36, pp. 46 595–46 623, 2023

2023

Cited alongside, same era.

X. She, Y. Liu, Y. Zhao, Y. He, L. Li, C. Tantithamthavorn, Z. Qin, and H. Wang, “Pitfalls in language models for code intelligence: A taxonomy and survey,” arXiv preprint arXiv:2310.17903 , 2023

Original

2023

Cited alongside, same era.

J. Liu, K. Wang, Y. Chen, X. Peng, Z. Chen, L. Zhang, and Y. Lou, “Large language model-based agents for software engineering: A survey,” arXiv preprint arXiv:2409.02977 , 2024

Original

2024

Cited alongside, same era.

Y. Ma, Q. Yang, R. Cao, B. Li, F. Huang, and Y. Li, “How to understand whole software repository?” arXiv preprint arXiv:2406.01422 , 2024

Original

2024

Closest in time.

W. Takerngsaksiri, C. Tantithamthavorn, and Y.-F. Li, “Syntax-aware on-the-fly code completion,” Information and Software Technology , vol. 165, p. 107336, 2024

2024

Closest in time.

W. Takerngsaksiri, C. Warusavitarne, C. Yaacoub, M. H. Keng Hou, and C. Tantithamthavorn, “Students’ perspectives on ai code completion: Benefits and challenges,” in 2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC) , 2024, pp. 1606–1611

2024

Closest in time.

W. Takerngsaksiri, R. Charakorn, C. Tantithamthavorn, and Y.-F. Li, “PyTester: Deep reinforcement learning for text-to-testcase generation,” arXiv preprint arXiv:2401.07576 , 2024

Original

2024

Closest in time.

S. Alagarsamy, C. Tantithamthavorn, and A. Aleti, “A3Test: Assertion-augmented automated test case generation,” Information and Software Technology , vol. 176, p. 107565, 2024

2024

Closest in time.

C. Pornprasit and C. Tantithamthavorn, “Fine-tuning and prompt engineering for large language models-based code review automation,” Information and Software Technology , 2024

2024

Closest in time.

Z. Li, S. Dutta, and M. Naik, “Llm-assisted static analysis for detecting security vulnerabilities,” arXiv preprint arXiv:2405.17238 , 2024

Original

2024

Closest in time.

M. Fu, C. Tantithamthavorn, T. Le, Y. Kume, V. Nguyen, D. Phung, and J. Grundy, “AIBugHunter: A practical tool for predicting, classifying and repairing software vulnerabilities.” Empirical Software Engineering , 2024

2024

Closest in time.

“Introducing swe-bench verified,” https://openai.com/index/introducing-swe-bench-verified/ , accessed: 2024-08-13

2024

Closest in time.

Y. Liu, T. Le-Cong, R. Widyasari, C. Tantithamthavorn, L. Li, X.-B. D. Le, and D. Lo, “Refining chatgpt-generated code: Characterizing and mitigating code quality issues,” ACM Transactions on Software Engineering and Methodology , vol. 33, no. 5, pp. 1–26, 2024

2024

Closest in time.