Large language models (LLMs) are becoming increasingly embedded in students' learning practices, yet much of what is known about how students use LLMs and how this usage impacts learning comes from problem-solving domains or constrained experimental settings. We present an analysis of data on LLM usage collected during two offerings of a research-oriented course where students learn to read, reason about, and critique academic papers. Without restrictions on whether or how to use LLMs, students reported their LLM usage practices when asked to do these activities as a series of homework assignments during the course. This paper extends prior work done on data from a single offering of the same course by presenting a refined bottom-up categorization of LLM usage types, cross-labeled by the extent of student initiative these usages entail. Furthermore, we examine how LLM use impacts student learning, measured by performance on three midterms, looking at factors such as frequency and type of usage.

翻译：大型语言模型（LLM）正日益融入学生的学习实践中，然而目前关于学生如何使用LLM及其对学习影响的认识，主要源自问题解决领域或受控实验环境。我们针对一门研究导向课程两届授课期间收集的LLM使用数据进行了分析，该课程要求学生学会阅读、推理并评议学术论文。在不限制LLM使用方式的情况下，学生在完成该课程系列作业时自主报告了其LLM使用实践。本文基于同一课程单届授课的先前研究数据，通过提出更精细的LLM使用类型自下而上分类法（并依据学生主动程度进行交叉标注）来扩展已有成果。此外，我们考察了LLM使用对学生学习的影响——以三次期中考试成绩为衡量指标，重点关注使用频率及类型等要素。