Knowledge tracing (KT) is the problem of predicting students' future performance based on their historical interactions with intelligent tutoring systems. Recently, many works present lots of special methods for applying deep neural networks to KT from different perspectives like model architecture, adversarial augmentation and etc., which make the overall algorithm and system become more and more complex. Furthermore, due to the lack of standardized evaluation protocol \citep{liu2022pykt}, there is no widely agreed KT baselines and published experimental comparisons become inconsistent and self-contradictory, i.e., the reported AUC scores of DKT on ASSISTments2009 range from 0.721 to 0.821 \citep{minn2018deep,yeung2018addressing}. Therefore, in this paper, we provide a strong but simple baseline method to deal with the KT task named \textsc{simpleKT}. Inspired by the Rasch model in psychometrics, we explicitly model question-specific variations to capture the individual differences among questions covering the same set of knowledge components that are a generalization of terms of concepts or skills needed for learners to accomplish steps in a task or a problem. Furthermore, instead of using sophisticated representations to capture student forgetting behaviors, we use the ordinary dot-product attention function to extract the time-aware information embedded in the student learning interactions. Extensive experiments show that such a simple baseline is able to always rank top 3 in terms of AUC scores and achieve 57 wins, 3 ties and 16 loss against 12 DLKT baseline methods on 7 public datasets of different domains. We believe this work serves as a strong baseline for future KT research. Code is available at \url{https://github.com/pykt-team/pykt-toolkit}\footnote{We merged our model to the \textsc{pyKT} benchmark at \url{https://pykt.org/}.}.
翻译:知识追踪(KT)问题旨在根据学生在智能教学系统中的历史交互行为预测其未来表现。近年来,大量研究从不同角度(如模型架构、对抗增强等)提出了应用深度神经网络解决知识追踪问题的专门方法,这使得整体算法与系统日趋复杂。此外,由于缺乏标准化评估协议(Liu等人,2022),目前尚不存在广泛认可的知识追踪基线,已发表的实验对比结果也呈现不一致甚至自相矛盾的现象——例如,DKT模型在ASSISTments2009数据集上报告的AUC分数范围可从0.721到0.821(Minn等人,2018;Yeung等人,2018)。因此,本文提出一种名为\textsc{simpleKT}的强健而简洁的基线方法来处理知识追踪任务。受心理测量学中拉希模型启发,我们显式建模题目特异性变异,以捕捉覆盖同一组知识组件(即学习者完成任务或问题步骤所需概念或技能的泛化术语)的题目间个体差异。此外,我们摒弃用于捕捉学生遗忘行为的复杂表示,直接采用普通点积注意力函数提取嵌入于学生学习交互中的时间感知信息。大量实验表明,这一简单基线在AUC分数上始终位列前三,并在涵盖不同领域的7个公开数据集上,对12种DLKT基线方法取得57胜3平16负的成绩。我们相信本研究将为未来知识追踪研究提供强基线。代码已开源至\url{https://github.com/pykt-team/pykt-toolkit}\footnote{我们已将模型合并至\textsc{pyKT}基准平台,网址为\url{https://pykt.org/}。}。