Large Language Models (LLMs) are increasingly utilized in educational tasks such as providing writing suggestions to students. Despite their potential, LLMs are known to harbor inherent biases which may negatively impact learners. Previous studies have investigated bias in models and data representations separately, neglecting the potential impact of LLM bias on human writing. In this paper, we investigate how bias transfers through an AI writing support pipeline. We conduct a large-scale user study with 231 students writing business case peer reviews in German. Students are divided into five groups with different levels of writing support: one classroom group with feature-based suggestions and four groups recruited from Prolific -- a control group with no assistance, two groups with suggestions from fine-tuned GPT-2 and GPT-3 models, and one group with suggestions from pre-trained GPT-3.5. Using GenBit gender bias analysis, Word Embedding Association Tests (WEAT), and Sentence Embedding Association Test (SEAT) we evaluate the gender bias at various stages of the pipeline: in model embeddings, in suggestions generated by the models, and in reviews written by students. Our results demonstrate that there is no significant difference in gender bias between the resulting peer reviews of groups with and without LLM suggestions. Our research is therefore optimistic about the use of AI writing support in the classroom, showcasing a context where bias in LLMs does not transfer to students' responses.

翻译：大型语言模型（LLM）越来越多地被应用于教育任务，例如为学生提供写作建议。尽管潜力巨大，但已知LLM存在固有偏见，可能对学习者产生负面影响。以往研究分别考察了模型和数据表示中的偏见，忽视了LLM偏见对人类写作的潜在影响。本文探究偏见如何通过AI写作支持流水线进行传递。我们开展了一项大规模用户研究，涉及231名用德语撰写商业案例同行评审的学生。学生被分为五组，接受不同水平的写作支持：一个课堂组获得基于特征的提示，其余四组从Prolific平台招募——包括一个无辅助的对照组、两组分别使用微调GPT-2和GPT-3模型生成的提示，以及一组使用预训练GPT-3.5模型生成的提示。通过GenBit性别偏见分析、词嵌入关联测试（WEAT）和句子嵌入关联测试（SEAT），我们评估了流水线各阶段的性别偏见：模型嵌入、模型生成的提示以及学生撰写的评审中。结果表明，接受与未接受LLM提示的群体在最终同行评审中的性别偏见无显著差异。因此，我们的研究对课堂中AI写作支持的使用持乐观态度，展示了一个LLM偏见并未传递至学生响应的情境。