A Preliminary Analysis on the Code Generation Capabilities of GPT-3.5 and Bard AI Models for Java Functions

Understand

This paper evaluates the capability of two state-of-the-art artificial intelligence (AI) models, GPT-3.5 and Bard, in generating Java code given a function description.

• We sourced the descriptions from CodingBat.com, a popular online platform that provides practice problems to learn programming.
• We compared the Java code generated by both models based on correctness, verified through the platform's own test cases.
• The results indicate clear differences in the capabilities of the two models.