English as foreign language_EFL_students' use of text generated from artificial intelligence_AI_natural language generation_NLG_tools may improve their writing quality. However, it remains unclear to what extent AI-generated text in these students' writing might lead to higher-quality writing. We explored 23 Hong Kong secondary school students' attempts to write stories comprising their own words and AI-generated text. Human experts scored the stories for dimensions of content, language and organization. We analyzed the basic organization and structure and syntactic complexity of the stories' AI-generated text and performed multiple linear regression and cluster analyses. The results show the number of human words and the number of AI-generated words contribute significantly to scores. Besides, students can be grouped into competent and less competent writers who use more AI-generated text or less AI-generated text compared to their peers. Comparisons of clusters reveal some benefit of AI-generated text in improving the quality of both high-scoring students' and low-scoring students' writing. The findings can inform pedagogical strategies to use AI-generated text for EFL students' writing and to address digital divides. This study contributes designs of NLG tools and writing activities to implement AI-generated text in schools.
翻译:英语作为外语(EFL)的学生使用人工智能自然语言生成(AI NLG)工具生成的文本,可能提高其写作质量。然而,这些学生写作中AI生成文本在多大程度上能带来更高质量的写作,目前尚不明确。我们探索了23名香港中学生尝试撰写包含自身词汇与AI生成文本的故事。人类专家从内容、语言和组织维度对故事进行评分。我们分析了故事中AI生成文本的基本组织结构与句法复杂度,并进行了多元线性回归和聚类分析。结果表明,人类词汇数量与AI生成词汇数量对评分有显著贡献。此外,学生可分为能力较强与较弱的两类写作者,他们相较于同龄人使用更多或更少的AI生成文本。聚类比较揭示了AI生成文本在提升高分学生与低分学生写作质量方面的某些益处。研究结果可为利用AI生成文本指导EFL学生写作及应对数字鸿沟提供教学策略。本研究为在学校中实施AI生成文本的NLG工具设计与写作活动贡献了方案。