Student modeling is central to many educational technologies as it enables predicting future learning outcomes and designing targeted instructional strategies. However, open-ended learning domains pose challenges for accurately modeling students due to the diverse behaviors and a large space of possible misconceptions. To approach these challenges, we explore the application of large language models (LLMs) for in-context student modeling in open-ended learning domains. More concretely, given a particular student's attempt on a reference task as observation, the objective is to synthesize the student's attempt on a target task. We introduce a novel framework, LLM for Student Synthesis (LLM-SS), that leverages LLMs for synthesizing a student's behavior. Our framework can be combined with different LLMs; moreover, we fine-tune LLMs to boost their student modeling capabilities. We instantiate several methods based on LLM-SS framework and evaluate them using an existing benchmark, StudentSyn, for student attempt synthesis in a visual programming domain. Experimental results show that our methods perform significantly better than the baseline method NeurSS provided in the StudentSyn benchmark. Furthermore, our method using a fine-tuned version of the GPT-3.5 model is significantly better than using the base GPT-3.5 model and gets close to human tutors' performance.
翻译:学生建模是众多教育技术的核心,它能够预测未来学习成果并设计有针对性的教学策略。然而,在开放式学习领域中,由于学生行为的多样性和潜在误解的巨大空间,准确建模学生面临挑战。为应对这些挑战,我们探索了大语言模型(LLMs)在开放式学习领域中进行情境学生建模的应用。具体而言,给定特定学生在参考任务上的尝试作为观察,目标是综合该学生在目标任务上的尝试。我们提出了一种新颖框架,即用于学生综合的LLM(LLM-SS),该框架利用大语言模型综合学生行为。我们的框架可与不同的大语言模型结合;此外,我们微调大语言模型以增强其学生建模能力。我们基于LLM-SS框架实例化了多种方法,并使用现有基准StudentSyn在可视化编程领域评估学生尝试综合的效果。实验结果表明,我们的方法显著优于StudentSyn基准中提供的基线方法NeurSS。此外,使用微调版GPT-3.5模型的方法显著优于基础版GPT-3.5模型,并接近人类导师的表现。