As a way of addressing increasingly sophisticated problems, software professionals face the constant challenge of seeking improvement. However, for these individuals to enhance their skills, their process of studying and training must involve feedback that is both immediate and accurate. In the context of software companies, where the scale of professionals undergoing training is large, but the number of qualified professionals available for providing corrections is small, delivering effective feedback becomes even more challenging. To circumvent this challenge, this work presents an exploration of using Large Language Models (LLMs) to support the correction process of open-ended questions in technical training. In this study, we utilized ChatGPT to correct open-ended questions answered by 42 industry professionals on two topics. Evaluating the corrections and feedback provided by ChatGPT, we observed that it is capable of identifying semantic details in responses that other metrics cannot observe. Furthermore, we noticed that, in general, subject matter experts tended to agree with the corrections and feedback given by ChatGPT.
翻译:为应对日益复杂的问题,软件专业人士面临着持续寻求改进的挑战。然而,要提升这些人员的技能,其学习与训练过程必须获得即时且准确的反馈。在软件公司背景下,当受训专业人员规模庞大而能提供纠正的合格专业人员数量有限时,提供有效反馈变得更具挑战性。为规避这一难题,本研究探索使用大语言模型(LLMs)支持技术培训中开放性问题的纠正过程。在本次研究中,我们利用ChatGPT对42名行业专业人员回答的两类主题开放性问题进行纠正。通过评估ChatGPT提供的纠正与反馈,我们观察到它能够识别其他指标无法观察到的回答中语义细节。此外,我们注意到主题专家总体上倾向于认同ChatGPT给出的纠正与反馈。