Where Is Self-admitted Code Generated by Large Language Models on GitHub?

Understand

The increasing use of Large Language Models (LLMs) in software development has garnered significant attention from researchers evaluating the capabilities and limitations of LLMs for code generation.

• However, much of the research focuses on controlled datasets such as HumanEval, which do not adequately capture the characteristics of LLM-generated code in real-world development scenarios.
• To address this gap, our study investigates self-admitted code generated by LLMs on GitHub, specifically focusing on instances where developers in projects with over five stars acknowledge the use of LLMs to generate code through code comments.
• Our findings reveal several key insights: (1) ChatGPT and Copilot dominate code generation, with minimal contributions from other LLMs.