On Evaluating the Efficiency of Source Code Generated by LLMs

Understand

Recent years have seen the remarkable capabilities of large language models (LLMs) for code generation.

• Different from existing work that evaluate the correctness of the code generated by LLMs, we propose to further evaluate its efficiency.
• More efficient code can lead to higher performance and execution efficiency of programs and software completed by LLM-assisted programming.
• First, we evaluate the efficiency of the code generated by LLMs on two benchmarks, HumanEval and MBPP.