With recent advances in generative AI, conversational models like ChatGPT have become feasible candidates for TAs. We investigate the practicality of using generative AI as TAs in introductory programming education by examining novice learners' interaction with TAs in a subgoal learning environment. To compare the learners' interaction and perception of the AI and human TAs, we conducted a between-subject study with 20 novice programming learners. Learners solve programming tasks by producing subgoals and subsolutions with the guidance of a TA. Our study shows that learners can solve tasks faster with comparable scores with AI TAs. Learners' perception of the AI TA is on par with that of human TAs in terms of speed and comprehensiveness of the replies and helpfulness, difficulty, and satisfaction of the conversation. Finally, we suggest guidelines to better design and utilize generative AI as TAs in programming education from the result of our chat log analysis.
翻译:随着生成式AI的最新进展,类似ChatGPT的对话模型已成为助教(TAs)的可行候选方案。我们通过考察初学者在子目标学习环境中与助教的互动,研究了在编程入门教育中将生成式AI用作助教的实用性。为了比较学习者在与AI助教和人类助教互动时的体验及感知,我们进行了一项涉及20名编程初学者的组间研究。学习者在助教指导下通过生成子目标和子解决方案来完成编程任务。我们的研究表明,使用AI助教的学习者能以相近的分数更快地完成任务。在回复速度、全面性以及对话的助益性、难度和满意度方面,学习者对AI助教的感知与对人类助教相当。最后,根据对话日志分析结果,我们提出了在编程教育中更好设计及利用生成式AI作为助教的指导原则。