The emergence of Large language models (LLMs) is expected to have a major impact on education. This paper explores the potential of using ChatGPT, an LLM, as a virtual Teaching Assistant (TA) in an Introductory Programming Course. We evaluate ChatGPT's capabilities by comparing its performance with that of human TAs in some TA functions. The TA functions which we focus on include (1) solving programming assignments, (2) grading student code submissions, and (3) providing feedback to undergraduate students in an introductory programming course. Firstly, we investigate how closely ChatGPT's solutions align with those submitted by students. This analysis goes beyond code correctness and also considers code quality. Secondly, we assess ChatGPT's proficiency in grading student code submissions using a given grading rubric and compare its performance with the grades assigned by human TAs. Thirdly, we analyze the quality and relevance of the feedback provided by ChatGPT. This evaluation considers how well ChatGPT addresses mistakes and offers suggestions for improvement in student solutions from both code correctness and code quality perspectives. We conclude with a discussion on the implications of integrating ChatGPT into computing education for automated grading, personalized learning experiences, and instructional support.
翻译:大型语言模型的出现预计将对教育产生重大影响。本文探讨了将ChatGPT(一种大型语言模型)用作入门编程课程中虚拟助教的潜力。我们通过将ChatGPT的表现与人类助教在某些助教职能中的表现进行比较来评估其能力。我们关注的助教职能包括:(1)解决编程作业问题;(2)对学生代码提交进行评分;(3)为入门编程课程的本科生提供反馈。首先,我们研究ChatGPT提供的解决方案与学生提交的方案之间的接近程度。该分析不仅关注代码正确性,还考虑代码质量。其次,我们评估ChatGPT根据给定评分标准对学生代码提交进行评分的熟练程度,并将其表现与人类助教分配的分数进行比较。最后,我们分析ChatGPT提供的反馈质量与相关性。这一评估考虑了ChatGPT在指出错误和提出改进建议方面(从代码正确性与代码质量两个角度)的能力。我们最后讨论了将ChatGPT整合到计算机教育中对于自动化评分、个性化学习体验以及教学支持的意义。