Knowledge tracing (KT) models are widely used to predict students' evolving knowledge states from their learning history. However, many KT models are evaluated using specific datasets, platforms, and learning contexts, raising questions about whether reported model performance replicates and generalizes across newer datasets that vary in context. This paper replicates and extends Zhang et al. (2021), which examined the cold-start problem in KT models and found that deep-learning-based KT models performed better, partly because of stronger predictions when students began practicing a skill. Using a more recent ASSISTments dataset, FoundationalASSIST, we replicate the previous analysis by evaluating model performance across opportunities to practice and extend the analysis by examining performance across problem types, including fill-in-the-blank, multiple-choice select-one, multiple-choice select-all, and order/sort problems. Results show that KT model performance varies across both student practice trajectories and problem types. Beyond the empirical replication, this study identifies practical challenges in reproducing educational data mining studies and serves as a proof of concept, showing how privacy-preserving research infrastructures such as SafeInsights can be leveraged to facilitate educational research and support replication analyses.

翻译：知识追踪（KT）模型广泛用于根据学生的学习历史预测其不断变化的知识状态。然而，许多KT模型是在特定数据集、平台和学习情境下进行评估的，这引发了关于所报告模型性能是否能在情境各异的新数据集中复现并泛化的疑问。本文对Zhang等人（2021）的研究进行了复现和扩展，该研究考察了KT模型中的冷启动问题，并发现基于深度学习的KT模型表现更佳，部分原因在于学生开始练习某项技能时预测能力更强。使用更新的ASSISTments数据集FoundationalASSIST，我们通过评估模型在不同练习机会下的性能复现了先前分析，并通过考察模型在不同题型（包括填空题、单选题、多选题以及排序题）上的表现扩展了分析范围。结果表明，KT模型的性能在学生练习轨迹和问题类型上都存在差异。除实证复现外，本研究还揭示了复现教育数据挖掘研究过程中面临的实际挑战，并作为概念验证，展示了SafeInsights等隐私保护型研究基础设施如何促进教育研究并支持复现分析。