Large Language Models (LLMs) have taken the world by storm, and students are assumed to use related tools at a great scale. In this research paper we aim to gain an understanding of how introductory programming students chat with LLMs and related tools, e.g., ChatGPT-3.5. To address this goal, computing students at a large German university were motivated to solve programming exercises with the assistance of ChatGPT as part of their weekly introductory course exercises. Then students (n=213) submitted their chat protocols (with 2335 prompts in sum) as data basis for this analysis. The data was analyzed w.r.t. the prompts, frequencies, the chats' progress, contents, and other use pattern, which revealed a great variety of interactions, both potentially supportive and concerning. Learning about students' interactions with ChatGPT will help inform and align teaching practices and instructions for future introductory programming courses in higher education.
翻译:大型语言模型(LLM)已席卷全球,学生群体被广泛认为正在大规模使用相关工具。本研究旨在探究入门编程学生如何与LLM及相关工具(例如ChatGPT-3.5)进行对话交流。为实现此目标,我们鼓励德国某大型高校的计算机专业学生在其每周的入门课程练习中借助ChatGPT完成编程习题。随后,学生(n=213)提交了他们的聊天记录(共计2335条提示)作为本分析的数据基础。研究从提示内容、频率、对话进程、主题及其他使用模式等维度对数据进行了分析,揭示了多样化的交互模式,其中既存在潜在的支持性互动,也包含值得关注的问题。了解学生与ChatGPT的交互方式,将有助于优化和调整高等教育中未来入门编程课程的教学实践与指导方案。