The use of natural language processing (NLP) techniques in engineering education can provide valuable insights into the underlying processes involved in generating text. While accessing these insights can be labor-intensive if done manually, recent advances in NLP and large language models have made it a realistic option for individuals. This study explores and evaluates a combination of clustering, summarization, and prompting techniques to analyze over 1,000 student essays in which students discussed their career interests. The specific assignment prompted students to define and explain their career goals as engineers. Using text embedding representations of student responses, we clustered the responses together to identify thematically similar statements from students. The clustered responses were then summarized to quickly identify career interest themes. We also used a set of a priori codes about career satisfaction and sectors to demonstrate an alternative approach to using these generative text models to analyze student writing. The results of this study demonstrate the feasibility and usefulness of NLP techniques in engineering education research. By automating the initial analysis of student essays, researchers and educators can more efficiently and accurately identify key themes and patterns in student writing. The methods presented in this paper have broader applications for engineering education and research purposes beyond analyzing student essays. By explaining these methods to the engineering education community, readers can utilize them in their own contexts.
翻译:自然语言处理(NLP)技术在工程教育中的应用,可为文本生成过程中的潜在机制提供宝贵见解。虽然手动获取这些见解可能耗费大量人力,但近年来NLP与大型语言模型的进展已使其成为个人可行的选择。本研究探索并评估了聚类、摘要和提示技术的组合应用,以分析1000余篇学生论述其职业兴趣的论文。该特定作业要求学生定义并解释其作为工程师的职业目标。通过使用学生回答的文本嵌入表征,我们对回答进行聚类以识别学生中主题相似的陈述。随后对聚类后的回答进行摘要,以快速识别职业兴趣主题。我们还使用了关于职业满意度和行业领域的先验编码集,展示使用这些生成式文本模型分析学生写作的替代方法。研究结果证明了NLP技术在工程教育研究中的可行性与实用性。通过自动化学生论文的初步分析,研究人员和教育工作者能够更高效、准确地识别学生写作中的关键主题与模式。本文提出的方法除分析学生论文外,在工程教育及研究领域具有更广泛的应用前景。通过向工程教育界阐释这些方法,读者可将其应用于自身的情境。