U.S. state education agencies mark schools displaying achievement gaps between demographic subgroups as needing improvement. Some schools may have few students in these subgroups, such that average end-of-year test scores only noisily measure the average "true" score--the score one would expect if students took the test many times. This, in addition to the masking of small subgroup averages in publicly available assessment data, poses challenges for evaluating interventions aimed at closing achievement gaps. We introduce propensity score estimates designed to achieve balance on subgroup average true scores. These estimates are available even when noisy measurements are not and improve overlap compared to those that ignore measurement error, leading to greater bias reduction of matching estimators. We demonstrate our methods through simulation and an application to a statewide initiative in Texas for curbing summer learning loss.
翻译:美国各州教育机构将存在人口统计亚组间学业成就差距的学校标记为需要改进。某些学校在这些亚组中可能学生数量较少,导致学年末平均测试成绩仅能嘈杂地反映平均'真实'成绩——即学生多次参加测试时预期获得的成绩。此外,公开评估数据中对小型亚组平均值的屏蔽,也给评估旨在缩小成就差距的干预措施带来了挑战。我们提出了旨在实现亚组平均真实成绩平衡的倾向得分估计方法。这些估计方法即使在无法获得噪声测量时仍可使用,并且相比忽略测量误差的方法能改善重叠性,从而提升匹配估计量的偏误削减效果。我们通过模拟研究及对德克萨斯州全州范围内遏制暑期学习损失计划的应用案例来验证所提方法。