Teaching students how to write code that is elegant, reusable, and comprehensible is a fundamental part of CS1 education. However, providing this "style feedback" in a timely manner has proven difficult to scale. In this paper, we present our experience deploying a novel, real-time style feedback tool in Code in Place, a large-scale online CS1 course. Our tool is based on the latest breakthroughs in large-language models (LLMs) and was carefully designed to be safe and helpful for students. We used our Real-Time Style Feedback tool (RTSF) in a class with over 8,000 diverse students from across the globe and ran a randomized control trial to understand its benefits. We show that students who received style feedback in real-time were five times more likely to view and engage with their feedback compared to students who received delayed feedback. Moreover, those who viewed feedback were more likely to make significant style-related edits to their code, with over 79% of these edits directly incorporating their feedback. We also discuss the practicality and dangers of LLM-based tools for feedback, investigating the quality of the feedback generated, LLM limitations, and techniques for consistency, standardization, and safeguarding against demographic bias, all of which are crucial for a tool utilized by students.

翻译：教导学生编写优雅、可复用且易读的代码是计算机科学导论（CS1）教育的核心环节。然而，如何规模化地及时提供这类"风格反馈"一直颇具挑战。本文报告我们在Code in Place（大规模在线CS1课程）中部署新型实时风格反馈工具的实践经验。该工具基于大型语言模型（LLMs）的最新突破，经过精心设计以确保对学生安全有益。我们在覆盖全球8000余名多元化学生的课堂中应用了实时风格反馈工具（RTSF），并通过随机对照实验验证其效能。研究表明，与接受延时反馈的学生相比，获得实时风格反馈的学生查看并参与反馈的可能性提升五倍。此外，查看反馈的学生更倾向于对代码进行实质性风格调整，其中79%的修改直接整合了反馈建议。本文还探讨了基于LLM的反馈工具的实用性与潜在风险，包括生成反馈的质量评估、LLM技术局限、一致性保障方法、标准化策略及人口统计偏见防范机制——这些对于学生使用的工具而言至关重要。