As a way of addressing increasingly sophisticated problems, software professionals face the constant challenge of seeking improvement. However, for these individuals to enhance their skills, their process of studying and training must involve feedback that is both immediate and accurate. In the context of software companies, where the scale of professionals undergoing training is large, but the number of qualified professionals available for providing corrections is small, delivering effective feedback becomes even more challenging. To circumvent this challenge, this work presents an exploration of using Large Language Models (LLMs) to support the correction process of open-ended questions in technical training. In this study, we utilized ChatGPT to correct open-ended questions answered by 42 industry professionals on two topics. Evaluating the corrections and feedback provided by ChatGPT, we observed that it is capable of identifying semantic details in responses that other metrics cannot observe. Furthermore, we noticed that, in general, subject matter experts tended to agree with the corrections and feedback given by ChatGPT.
翻译:为应对日益复杂的问题,软件专业人员始终面临提升自身能力的挑战。然而,这类人员若要精进技能,其学习与训练过程必须依托既即时又精准的反馈。在软件企业环境中,接受培训的专业人员规模庞大,而能提供纠偏指导的合格专家数量有限,这使得提供有效反馈变得更加困难。为破解这一难题,本研究探索了利用大语言模型(LLMs)辅助技术培训中开放式试题批改过程的可行性。我们使用ChatGPT对42名行业从业人员就两个主题作答的开放式试题进行批改。通过评估ChatGPT提供的批改结果与反馈,发现该模型能够识别其他测评指标无法捕捉的语义细节。此外,我们注意到领域专家普遍认同ChatGPT给出的批改意见与反馈。