Despite growing interest in using LLMs to generate feedback on students' writing, little is known about how students respond to AI-mediated versus human-provided feedback. We address this gap through a randomized controlled trial in a large introductory economics course (N=354), where we introduce and deploy FeedbackWriter - a system that generates AI suggestions to teaching assistants (TAs) while they provide feedback on students' knowledge-intensive essays. TAs have the full capacity to adopt, edit, or dismiss the suggestions. Students were randomly assigned to receive either handwritten feedback from TAs (baseline) or AI-mediated feedback where TAs received suggestions from FeedbackWriter. Students revise their drafts based on the feedback, which is further graded. In total, 1,366 essays were graded using the system. We found that students receiving AI-mediated feedback produced significantly higher-quality revisions, with gains increasing as TAs adopted more AI suggestions. TAs found the AI suggestions useful for spotting gaps and clarifying rubrics.

翻译：尽管利用大语言模型生成学生写作反馈的兴趣日益增长，但学生对 AI 介导反馈与人工反馈的回应方式尚不明确。本研究通过在一门大型经济学导论课程（N=354）中进行的随机对照试验填补了这一空白。我们引入并部署了 FeedbackWriter——一个在助教为学生知识密集型论文提供反馈时生成 AI 建议的系统。助教拥有完全自主权采纳、编辑或忽略这些建议。学生被随机分配接收助教的手写反馈（基线组）或接收助教通过 FeedbackWriter 获得建议后形成的 AI 介导反馈。学生根据反馈修订草稿，修订稿将进一步接受评分。该系统共计评阅了 1,366 篇论文。研究发现，接收 AI 介导反馈的学生产出了质量显著更高的修订稿，且当助教采纳更多 AI 建议时，质量提升更为明显。助教认为 AI 建议有助于发现内容漏洞和厘清评分标准。