Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis

Understand

In the first half of 2023, text-generative artificial intelligence (AI), including ChatGPT, equipped with GPT-3.5 and GPT-4, from OpenAI, has attracted considerable attention worldwide.

• In this study, first, we compared Japanese stylometric features of texts generated by GPT (-3.5 and -4) and those written by humans.
• In this work, we performed multi-dimensional scaling (MDS) to confirm the distributions of 216 texts of three classes (72 academic papers written by 36 single authors, 72 texts generated by GPT-3.5, and 72 texts generated by GPT-4 on the basis of the titles of the aforementioned papers) focusing on the following stylometric features: (1) bigrams of parts-of-speech, (2) bigram of postpositional particle words, (3) positioning of commas, and (4) rate of function words.
• MDS revealed distinct distributions at each stylometric feature of GPT (-3.5 and -4) and human.