ChatGPT is a publicly available chatbot that can quickly generate texts on given topics, but it is unknown whether the chatbot is really superior to human writers in all aspects of writing and whether its writing quality can be prominently improved on the basis of updating commands. Consequently, this study compared the writing performance on a narrative topic by ChatGPT and Chinese intermediate English (CIE) learners so as to reveal the chatbot's advantage and disadvantage in writing. The data were analyzed in terms of five discourse components using Coh-Metrix (a special instrument for analyzing language discourses), and the results revealed that ChatGPT performed better than human writers in narrativity, word concreteness, and referential cohesion, but worse in syntactic simplicity and deep cohesion in its initial version. After more revision commands were updated, while the resulting version was facilitated in syntactic simplicity, yet it is still lagged far behind CIE learners' writing in deep cohesion. In addition, the correlation analysis of the discourse components suggests that narrativity was correlated with referential cohesion in both ChatGPT and human writers, but the correlations varied within each group.

翻译：ChatGPT是一款可公开获取的聊天机器人，能针对给定主题快速生成文本，但尚不清楚该聊天机器人是否在所有写作维度上均优于人类写作者，亦不明晰其写作质量能否通过更新指令获得显著提升。为此，本研究通过比较ChatGPT与中文中级英语学习者在叙事性写作任务中的表现，揭示该聊天机器人在写作中的优势与不足。研究采用Coh-Metrix（一种专门分析语言语篇的工具）对数据从五个语篇成分维度进行分析，结果显示：初始版本的ChatGPT在叙事性、词汇具体性和指代衔接上表现优于人类写作者，但在句法简单性和深层衔接上表现较差；在更新更多修改指令后，新版ChatGPT在句法简单性上有所改善，但其深层衔接仍远落后于中文中级英语学习者的写作。此外，对语篇成分的相关性分析表明，叙事性与指代衔接在ChatGPT和人类写作者中均存在相关性，但各群体内部的相关关系存在差异。