The growing enrollments in computer science courses and increase in class sizes necessitate scalable, automated tutoring solutions to adequately support student learning. While Large Language Models (LLMs) like GPT-4 have demonstrated potential in assisting students through question-answering, educators express concerns over student overreliance, miscomprehension of generated code, and the risk of inaccurate answers. Rather than banning these tools outright, we advocate for a constructive approach that harnesses the capabilities of AI while mitigating potential risks. This poster introduces CourseAssist, a novel LLM-based tutoring system tailored for computer science education. Unlike generic LLM systems, CourseAssist uses retrieval-augmented generation, user intent classification, and question decomposition to align AI responses with specific course materials and learning objectives, thereby ensuring pedagogical appropriateness of LLMs in educational settings. We evaluated CourseAssist against a baseline of GPT-4 using a dataset of 50 question-answer pairs from a programming languages course, focusing on the criteria of usefulness, accuracy, and pedagogical appropriateness. Evaluation results show that CourseAssist significantly outperforms the baseline, demonstrating its potential to serve as an effective learning assistant. We have also deployed CourseAssist in 6 computer science courses at a large public R1 research university reaching over 500 students. Interviews with 20 student users show that CourseAssist improves computer science instruction by increasing the accessibility of course-specific tutoring help and shortening the feedback loop on their programming assignments. Future work will include extensive pilot testing at more universities and exploring better collaborative relationships between students, educators, and AI that improve computer science learning experiences.

翻译：计算机科学课程注册人数的增长与班级规模的扩大，亟需可扩展的自动化辅导方案以充分支持学生学习。尽管GPT-4等大语言模型在通过问答辅助学生方面展现出潜力，但教育工作者担忧学生可能过度依赖、误解生成代码，以及存在答案不准确的风险。我们主张采取建设性方法而非直接禁用这些工具，在利用人工智能能力的同时降低潜在风险。本海报介绍CourseAssist——一种专为计算机科学教育定制的新型大语言模型辅导系统。与通用大语言模型系统不同，CourseAssist采用检索增强生成、用户意图分类和问题分解技术，使AI响应与具体课程材料及学习目标保持一致，从而确保大语言模型在教育环境中的教学适配性。我们使用编程语言课程的50组问答对数据集，从实用性、准确性和教学适配性三个维度，将CourseAssist与GPT-4基线模型进行对比评估。评估结果表明CourseAssist显著优于基线模型，证明其具备成为有效学习助手的潜力。我们已在一所大型公立R1研究型大学的6门计算机科学课程中部署CourseAssist，覆盖超过500名学生。对20名学生用户的访谈显示，CourseAssist通过提升课程专属辅导资源的可及性、缩短编程作业的反馈周期，有效改善了计算机科学教学。未来工作将包括在更多高校开展广泛试点测试，并探索学生、教育者与AI之间能提升计算机科学学习体验的协同关系。