Background: Large Language Models (LLMs) such as ChatGPT and CoPilot are influencing software engineering practice. Software engineering educators must teach future software engineers how to use such tools well. As of yet, there have been few studies that report on the use of LLMs in the classroom. It is, therefore, important to evaluate students' perception of LLMs and possible ways of adapting the computing curriculum to these shifting paradigms. Purpose: The purpose of this study is to explore computing students' experiences and approaches to using LLMs during a semester-long software engineering project. Design/Method: We collected data from a senior-level software engineering course at Purdue University. This course uses a project-based learning (PBL) design. The students used LLMs such as ChatGPT and Copilot in their projects. A sample of these student teams were interviewed to understand (1) how they used LLMs in their projects; and (2) whether and how their perspectives on LLMs changed over the course of the semester. We analyzed the data to identify themes related to students' usage patterns and learning outcomes. Results/Discussion: When computing students utilize LLMs within a project, their use cases cover both technical and professional applications. In addition, these students perceive LLMs to be efficient tools in obtaining information and completion of tasks. However, there were concerns about the responsible use of LLMs without being detrimental to their own learning outcomes. Based on our findings, we recommend future research to investigate the usage of LLM's in lower-level computer engineering courses to understand whether and how LLMs can be integrated as a learning aid without hurting the learning outcomes.
翻译:背景:ChatGPT与CoPilot等大语言模型(LLMs)正深刻影响着软件工程实践。软件工程教育者必须教会未来的软件工程师如何善用此类工具。目前,关于LLMs在课堂中应用的实证研究尚属少数。因此,评估学生对LLMs的认知以及计算机课程体系适应这种范式转变的可行路径具有重要价值。目的:本研究旨在探究计算机专业学生在为期一学期的软件工程项目中使用LLMs的实践体验与方法策略。设计/方法:我们采集了普渡大学高年级软件工程课程的教学数据。该课程采用基于项目的学习(PBL)模式,学生在项目中使用了ChatGPT、Copilot等LLMs工具。研究团队选取学生团队样本进行访谈,以了解:(1)他们如何在项目中应用LLMs;(2)他们对LLMs的看法是否以及如何在学期过程中发生变化。通过分析数据,我们提炼出与学生使用模式和学习成效相关的主题。结果/讨论:当计算机专业学生在项目中运用LLMs时,其应用场景涵盖技术应用与专业实践两个维度。此外,这些学生认为LLMs是获取信息与完成任务的高效工具。但学生们也担忧,如何在不妨碍自身学习成果的前提下负责任地使用LLMs。基于研究发现,我们建议未来研究应探索LLMs在低年级计算机工程课程中的应用,以明确LLMs能否以及如何在不损害学习效果的前提下作为学习辅助工具融入教学。