Despite growing interest in using LLMs to generate feedback on students' writing, little is known about how students respond to AI-mediated versus human-provided feedback. We address this gap through a randomized controlled trial in a large introductory economics course (N=354), where we introduce and deploy FeedbackWriter - a system that generates AI suggestions to teaching assistants (TAs) while they provide feedback on students' knowledge-intensive essays. TAs have the full capacity to adopt, edit, or dismiss the suggestions. Students were randomly assigned to receive either handwritten feedback from TAs (baseline) or AI-mediated feedback where TAs received suggestions from FeedbackWriter. Students revise their drafts based on the feedback, which is further graded. In total, 1,366 essays were graded using the system. We found that students receiving AI-mediated feedback produced significantly higher-quality revisions, with gains increasing as TAs adopted more AI suggestions. TAs found the AI suggestions useful for spotting gaps and clarifying rubrics.

翻译：尽管利用大型语言模型（LLM）为学生写作提供反馈的兴趣日益增长，但学生对AI辅助反馈与人工反馈的回应方式尚不明确。本研究通过在一门大型经济学导论课程（N=354）中开展的随机对照试验填补这一空白。我们引入并部署了FeedbackWriter系统——该系统能在助教（TA）批改学生知识密集型论文时为其生成AI建议，助教可完全自主选择采纳、修改或忽略这些建议。学生被随机分配接收两种反馈：助教手写反馈（基线组）或AI辅助反馈（助教通过FeedbackWriter接收建议组）。学生根据反馈修改初稿后提交修订版接受评分，系统共完成1,366篇论文的评分。研究发现：接收AI辅助反馈的学生修订稿质量显著更高，且当助教采纳更多AI建议时，质量提升效果更为明显。助教反馈显示，AI建议有助于发现内容漏洞并厘清评分标准。