Providing personalized assistance at scale is a long-standing challenge for computing educators, but a new generation of tools powered by large language models (LLMs) offers immense promise. Such tools can, in theory, provide on-demand help in large class settings and be configured with appropriate guardrails to prevent misuse and mitigate common concerns around learner over-reliance. However, the deployment of LLM-powered tools in authentic classroom settings is still rare, and very little is currently known about how students will use them in practice and what type of help they will seek. To address this, we examine students' use of an innovative LLM-powered tool that provides on-demand programming assistance without revealing solutions directly. We deployed the tool for 12 weeks in an introductory computer and data science course ($n = 52$), collecting more than 2,500 queries submitted by students throughout the term. We manually categorized all student queries based on the type of assistance sought, and we automatically analyzed several additional query characteristics. We found that most queries requested immediate help with programming assignments, whereas fewer requests asked for help on related concepts or for deepening conceptual understanding. Furthermore, students often provided minimal information to the tool, suggesting this is an area in which targeted instruction would be beneficial. We also found that students who achieved more success in the course tended to have used the tool more frequently overall. Lessons from this research can be leveraged by programming educators and institutions who plan to augment their teaching with emerging LLM-powered tools.

翻译：为大规模课堂提供个性化辅导一直是计算机教育工作者面临的长期挑战，而由大语言模型（LLM）驱动的新一代工具展现出巨大潜力。理论上，这类工具能够在大班教学中按需提供帮助，并通过设置适当的防护措施防止滥用，同时缓解学习者过度依赖等常见问题。然而，LLM工具在真实课堂环境中的部署仍十分罕见，目前对其实际使用模式及学生寻求帮助的类型知之甚少。为此，我们研究了学生对一款创新性LLM工具的使用情况——该工具在不直接显示解决方案的前提下提供按需编程辅导。我们在为期12周的计算机与数据科学入门课程中部署该工具（n=52），收集了学生整个学期提交的超过2500条查询。我们基于寻求帮助的类型对所有学生查询进行了人工分类，并自动分析了多项附加查询特征。研究发现，大部分查询旨在寻求编程作业的即时帮助，而较少涉及相关概念或深化概念理解的需求。此外，学生向工具提供的信息往往极为简略，这表明针对性地开展指导将大有裨益。我们还发现，课程表现更优异的学生总体使用该工具的频次更高。编程教育工作者和计划借助新兴LLM工具辅助教学的机构，可借鉴本研究的经验教训。