Large Language Models (LLMs) represent a leap in artificial intelligence, excelling in tasks using human language(s). Although the main focus of general-purpose LLMs is not code generation, they have shown promising results in the domain. However, the usefulness of LLMs in an academic software engineering project has not been fully explored yet. In this study, we explore the usefulness of LLMs for 214 students working in teams consisting of up to six members. Notably, in the academic course through which this study is conducted, students were encouraged to integrate LLMs into their development tool-chain, in contrast to most other academic courses that explicitly prohibit the use of LLMs. In this paper, we analyze the AI-generated code, prompts used for code generation, and the human intervention levels to integrate the code into the code base. We also conduct a perception study to gain insights into the perceived usefulness, influencing factors, and future outlook of LLM from a computer science student's perspective. Our findings suggest that LLMs can play a crucial role in the early stages of software development, especially in generating foundational code structures, and helping with syntax and error debugging. These insights provide us with a framework on how to effectively utilize LLMs as a tool to enhance the productivity of software engineering students, and highlight the necessity of shifting the educational focus toward preparing students for successful human-AI collaboration.

翻译：大语言模型代表了人工智能的飞跃，在执行利用人类语言的任务中表现出色。尽管通用大语言模型的主要关注点并非代码生成，但它们在该领域已显示出令人期待的结果。然而，在学术软件工程项目中大语言模型的效用尚未得到充分探索。在本研究中，我们探究了大语言模型对214名以最多六人团队形式工作的学生的效用。值得注意的是，在开展本研究的学术课程中，鼓励学生将大语言模型整合到其开发工具链中，这与大多数明令禁止使用大语言模型的其他学术课程形成对比。本文分析了AI生成的代码、用于代码生成的提示词，以及将代码集成到代码库中的人工干预程度。我们还进行了一项认知研究，以从计算机科学学生的视角深入了解大语言模型的感知效用、影响因素及未来展望。我们的研究结果表明，大语言模型在软件开发的早期阶段可发挥关键作用，尤其是在生成基础代码结构以及辅助语法和错误调试方面。这些见解为我们提供了一个框架，说明如何有效利用大语言模型作为提升软件工程学生生产力的工具，并凸显了将教育重点转向培养学生成功进行人机协作的必要性。