Toward Functional and Non-Functional Evaluation of Application-Level Code Generation

Understand

Large language models (LLMs) have achieved strong performance on code generation.

• However, most prior evaluations focus on snippet-level outputs, such as function generation or repository completion.
• These settings do not fully evaluate application-level code generation, where the goal is to produce a runnable repository with coherent multi-file structure, dependency support, and end-to-end executability.
• In addition, real-world software quality depends not only on functional correctness but also on non-functional quality attributes, such as maintainability and security.