Knowledge tracing (KT) models are widely used to predict students' evolving knowledge states from their learning history. However, many KT models are evaluated using specific datasets, platforms, and learning contexts, raising questions about whether reported model performance replicates and generalizes across newer datasets that vary in context. This paper replicates and extends Zhang et al. (2021), which examined the cold-start problem in KT models and found that deep-learning-based KT models performed better, partly because of stronger predictions when students began practicing a skill. Using a more recent ASSISTments dataset, FoundationalASSIST, we replicate the previous analysis by evaluating model performance across opportunities to practice and extend the analysis by examining performance across problem types, including fill-in-the-blank, multiple-choice select-one, multiple-choice select-all, and order/sort problems. Results show that KT model performance varies across both student practice trajectories and problem types. Beyond the empirical replication, this study identifies practical challenges in reproducing educational data mining studies and serves as a proof of concept, showing how privacy-preserving research infrastructures such as SafeInsights can be leveraged to facilitate educational research and support replication analyses.

翻译：知识追踪（KT）模型广泛用于根据学生的学习历史预测其不断变化的知识状态。然而，许多KT模型是在特定数据集、平台和学习情境下进行评估的，这引发了一个疑问：所报告模型性能是否能在情境各异的新数据集中复现并泛化。本文对张等人（2021）的研究进行了复现与扩展，该研究探讨了KT模型中的冷启动问题，并发现基于深度学习的KT模型表现更佳，部分原因在于当学生开始练习某一技能时，这类模型能做出更强预测。我们利用更新的ASSISTments数据集FoundationalASSIST，通过评估模型在不同练习机会中的表现来复现先前的分析，并通过考察模型在填空题、单选题、多选题及排序题等不同题型上的表现来扩展分析。结果表明，KT模型的性能既随学生练习轨迹变化，也因题型而异。除实证复现外，本研究还识别了复现教育数据挖掘研究中的实际挑战，并作为概念验证，展示了SafeInsights等隐私保护研究基础设施如何被用于促进教育研究并支持复现分析。