A Comparative Analysis of Large Language Models for Code Documentation Generation

Analysis of variance (ANOVA)

Lars St»hle and Svante Wold. 1989 · 1989

Earlier work this paper cites.

How software engineers use documentation: the state of the practice

T.C. Lethbridge, J. Singer, and A. Forward. 2003 · 2003

Earlier work this paper cites.

Language Models are Few-Shot Learners

Original

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2005

Earlier work this paper cites.

Interrater agreement and interrater reliability: Key concepts, approaches, and applications

Natasa Gisev, J. Simon Bell, and Timothy F. Chen. 2013 · 2012

Earlier work this paper cites.

To document or not to document? An exploratory study on developers’ motivation to document code. In Advanced Information Systems Engineering Workshops: CAiSE 2015 International Workshops, Stockholm, Sweden, June 8-9, 2015, Proceedings 27 . Springer, 100–106

Yulia Shmerlin, Irit Hadar, Doron Kliger, and Hayim Makabee. 2015 · 2015

Earlier work this paper cites.

Automatic Code Documentation Generation Using GPT-3. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering . ACM, Rochester MI USA, 1–6

Junaed Younus Khan and Gias Uddin. 2022 · 2022

Earlier work this paper cites.

Generating Diverse Code Explanations using the GPT-3 Large Language Model. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2 (ICER ’22, Vol. 2) . Association for Computing Machinery, New York, NY, USA, 37–39

Stephen MacNeil, Andrew Tran, Dan Mogil, Seth Bernstein, Erin Ross, and Ziheng Huang. 2022 · 2022

Earlier work this paper cites.

A Review on Source Code Documentation

Sawan Rai, Ramesh Chandra Belwal, and Atul Gupta. 2022 · 2022

Cited alongside, same era.

Few-shot training LLMs for project-specific code-summarization. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE ’22) . Association for Computing Machinery, New York, NY, USA, 1–5

Toufique Ahmed and Premkumar Devanbu. 2023 · 2023

Cited alongside, same era.

Open-Source Large Language Models Outperform Crowd Workers and Approach ChatGPT in Text-Annotation Tasks

Original

Meysam Alizadeh, Maël Kubli, Zeynab Samei, Shirin Dehghani, Juan Diego Bermeo, Maria Korobeynikova, and Fabrizio Gilardi. 2023 · 2023

Cited alongside, same era.

Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learning

Original

Mingyang Geng, Shangwen Wang, Dezun Dong, Haotian Wang, Ge Li, Zhi Jin, Xiaoguang Mao, and Xiangke Liao. 2023 · 2023

Cited alongside, same era.

GPT-4 Technical Report

Original

OpenAI. 2023 · 2023

Closest in time.

The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23) . Association for Computing Machinery, New York, NY, USA, 491–514

Steven I. Ross, Fernando Martinez, Stephanie Houde, Michael Muller, and Justin D. Weisz. 2023 · 2023

Closest in time.

HotGPT: How to Make Software Documentation More Useful with a Large Language Model?. In Proceedings of the 19th Workshop on Hot Topics in Operating Systems (HOTOS ’23) . Association for Computing Machinery, New York, NY, USA, 87–93

Yiming Su, Chengcheng Wan, Utsav Sethi, Shan Lu, Madan Musuvathi, and Suman Nath. 2023 · 2023

Closest in time.

Llama 2: Open Foundation and Fine-Tuned Chat Models

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023 · 2023