Knowledge tracing (KT) is the problem of predicting students' future performance based on their historical interactions with intelligent tutoring systems. Recently, many works present lots of special methods for applying deep neural networks to KT from different perspectives like model architecture, adversarial augmentation and etc., which make the overall algorithm and system become more and more complex. Furthermore, due to the lack of standardized evaluation protocol \citep{liu2022pykt}, there is no widely agreed KT baselines and published experimental comparisons become inconsistent and self-contradictory, i.e., the reported AUC scores of DKT on ASSISTments2009 range from 0.721 to 0.821 \citep{minn2018deep,yeung2018addressing}. Therefore, in this paper, we provide a strong but simple baseline method to deal with the KT task named \textsc{simpleKT}. Inspired by the Rasch model in psychometrics, we explicitly model question-specific variations to capture the individual differences among questions covering the same set of knowledge components that are a generalization of terms of concepts or skills needed for learners to accomplish steps in a task or a problem. Furthermore, instead of using sophisticated representations to capture student forgetting behaviors, we use the ordinary dot-product attention function to extract the time-aware information embedded in the student learning interactions. Extensive experiments show that such a simple baseline is able to always rank top 3 in terms of AUC scores and achieve 57 wins, 3 ties and 16 loss against 12 DLKT baseline methods on 7 public datasets of different domains. We believe this work serves as a strong baseline for future KT research. Code is available at \url{https://github.com/pykt-team/pykt-toolkit}\footnote{We merged our model to the \textsc{pyKT} benchmark at \url{https://pykt.org/}.}.
翻译:知识追踪(KT)旨在根据学生在智能辅导系统中的历史交互记录预测其未来表现。近年来,诸多研究从模型架构、对抗增强等不同角度提出了大量将深度神经网络应用于知识追踪的特殊方法,这使得整体算法与系统日趋复杂。此外,由于缺乏标准化评估协议(\citep{liu2022pykt}),目前尚无广泛公认的知识追踪基线,已发表的实验对比结果呈现不一致甚至自相矛盾的现象——例如,DKT在ASSISTments2009数据集上的报告AUC得分介于0.721至0.821之间(\citep{minn2018deep,yeung2018addressing})。为此,本文提出一种名为\textsc{simpleKT}的强健而简洁的基线方法处理知识追踪任务。受心理学测量中Rasch模型的启发,我们显式建模问题特异性变异,以捕捉覆盖同一知识组件集合(即学习者完成任务或解决问题所需概念或技能的泛化术语)的题目之间的个体差异。此外,我们未采用复杂表征捕捉学生遗忘行为,而是利用普通点积注意力函数提取嵌入学生学习交互中的时间感知信息。大量实验表明,这一简洁基线方法在不同领域的7个公开数据集上始终能够获得AUC得分前三的排名,并在与12种深度学习知识追踪基线方法的对比中取得57胜、3平、16负的成绩。我们相信该工作可为未来知识追踪研究提供强有力的基线。代码已开源在\url{https://github.com/pykt-team/pykt-toolkit}\footnote{我们已将模型整合至\textsc{pyKT}基准平台\url{https://pykt.org/}。}。