Mixed-effects models fit to observational practice data are widely used in learning analytics to estimate student-level variation in initial knowledge and learning rate, and the resulting estimates increasingly inform substantive claims about learners. We examine whether such estimates can be read as properties of learners or whether they depend on choices about which observations the model is fit to. As a case study, we revisit the ``astonishing regularity'' reported by Koedinger et al. (2023): that students vary substantially in initial knowledge but much less in learning rate. The finding is based on fits of the individual Additive Factors Model (iAFM) to 27 educational datasets, and rests on a model-derived estimate of student-level learning-rate variation being small in absolute terms. We refit the same model on the same datasets under two specifications, each varying how much of each student's practice on a given skill is used in fitting. The estimate of student-level variation in initial knowledge stays approximately stable across both specifications. The estimate of student-level variation in learning rate does not: it inflates by a median of 118\% under one specification and is several times larger under the other. The same model, fit to the same data, returns substantially different estimates of how much students vary in learning rate depending on which observations are included. When estimates from mixed-effects models on observational practice data are used to support substantive claims about learners, sensitivity to such choices deserves a central place in how those estimates are reported and read.

翻译：基于观测性练习数据的混合效应模型广泛应用于学习分析中，用于估计学生在初始知识和学习率方面的个体差异，这些估计结果越来越多地被用于支持关于学习者的实质性论断。我们探究了此类估计是否应被视为学习者的固有属性，抑或取决于模型所拟合的观测数据选择。作为案例研究，我们重新审视了Koedinger等人（2023）报告的"惊人规律性"：学生在初始知识方面存在显著差异，但在学习率方面差异较小。该发现基于对27个教育数据集拟合个体加性因子模型（iAFM）的结果，并依赖于模型导出的学生层面学习率变异估计在绝对值上较小这一结论。我们在两种设定下对相同数据集重新拟合同一模型，每种设定分别改变学生在特定技能上的练习数据纳入量。初始知识的学生层面变异估计在两种设定中保持基本稳定，但学习率的学生层面变异估计则不然：在一种设定下中位数膨胀118%，在另一种设定下则扩大数倍。同一模型、相同数据，仅因观测值选择不同，便得出学生间学习率差异截然不同的估计结果。当基于观测性练习数据的混合效应模型估计被用于支持关于学习者的实质性论断时，此类选择敏感性应在报告和解读这些估计时占据核心地位。