A transformation called normalized gain (ngain) has been acknowledged as one of the most common measures of knowledge growth in pretest-posttest contexts in physics education research. Recent studies in math education have shown that ngains can also be applied to assess learners' ability to acquire unfamiliar knowledge, that is, to estimate their "learning rate". This quantity is estimated from learning data through two well-known methods: computing the average ngain of the group or computing the ngain of the average learner. These two methods commonly yield different results, and prior research has concluded that the difference between them is associated with a pretest-ngains correlation. Such a correlation would suggest a bias of this learning measurement because it implies its favoring of certain subgroups of students according to their performance in pretest measurements. The present study analyzes these two estimation methods by drawing on statistical models. Our results show that the two estimation methods are equivalent when no measurement errors exist. In contrast, when there are measurement errors, the first method provides a biased estimator, whereas the second one provides an unbiased estimator. Furthermore, these measurement errors induce a spurious correlation between the pretest and ngain scores. Our results seem consistent with prior research, except they show that measurement errors in pretest and posttest scores are the source of a spurious pretest-ngain correlation. Consequently, estimating learning rates might effectively provide unbiased estimates of knowledge change that control for the effect of prior knowledge even in the presence of pretest-ngain correlations.
翻译:在物理教育研究中,一种称为归一化增益(ngain)的变换已被公认为前测-后测情境下衡量知识增长最常用的指标之一。近期数学教育研究表明,归一化增益也可用于评估学习者获取陌生知识的能力,即估计其“学习速率”。该量值可通过两种常用方法从学习数据中估算:计算群体的平均归一化增益,或计算平均学习者的归一化增益。这两种方法通常得出不同结果,先前研究认为其差异与前测-归一化增益相关性有关。这种相关性可能暗示该学习测量存在偏差,因为它意味着测量结果会根据学生在预测试中的表现而偏向特定学生子群体。本研究借助统计模型对这两种估计方法进行分析。结果表明,在不存在测量误差时,两种估计方法是等价的。相反,当存在测量误差时,第一种方法会产生有偏估计量,而第二种方法则提供无偏估计量。此外,这些测量误差会引发前测分数与归一化增益分数之间的伪相关。我们的结果与先前研究基本一致,但揭示了前测与后测分数的测量误差才是产生伪前测-归一化增益相关的根源。因此,即使在存在前测-归一化增益相关的情况下,估计学习速率仍可能有效提供控制先验知识影响的无偏知识变化估计。