Early warning systems (EWS) are predictive tools at the center of recent efforts to improve graduation rates in public schools across the United States. These systems assist in targeting interventions to individual students by predicting which students are at risk of dropping out. Despite significant investments in their widespread adoption, there remain large gaps in our understanding of the efficacy of EWS, and the role of statistical risk scores in education. In this work, we draw on nearly a decade's worth of data from a system used throughout Wisconsin to provide the first large-scale evaluation of the long-term impact of EWS on graduation outcomes. We present empirical evidence that the prediction system accurately sorts students by their dropout risk. We also find that it may have caused a single-digit percentage increase in graduation rates, though our empirical analyses cannot reliably rule out that there has been no positive treatment effect. Going beyond a retrospective evaluation of DEWS, we draw attention to a central question at the heart of the use of EWS: Are individual risk scores necessary for effectively targeting interventions? We propose a simple mechanism that only uses information about students' environments -- such as their schools, and districts -- and argue that this mechanism can target interventions just as efficiently as the individual risk score-based mechanism. Our argument holds even if individual predictions are highly accurate and effective interventions exist. In addition to motivating this simple targeting mechanism, our work provides a novel empirical backbone for the robust qualitative understanding among education researchers that dropout is structurally determined. Combined, our insights call into question the marginal value of individual predictions in settings where outcomes are driven by high levels of inequality.

翻译：早期预警系统（EWS）是美国公立学校近年来提升毕业率的核心预测工具。这类系统通过预测哪些学生存在辍学风险，协助将干预措施精准定位至个体学生。尽管在全面推广中投入了大量资源，但我们对EWS有效性及其统计风险评分在教育中作用的理解仍存在重大空白。本研究利用威斯康星州范围内持续近十年的系统数据，首次对EWS对毕业结果的长期影响进行大规模评估。实证证据表明，该预测系统能准确按辍学风险对学生进行分层。同时我们发现，该系统可能使毕业率提升了个位数百分比，但实证分析尚无法可靠排除零处理效应的可能性。在超越对DEWS回顾性评估的基础上，我们聚焦于EWS应用的核心问题：精准干预是否必须依赖个体风险评分？本文提出一种仅利用学生环境信息（如所在学校和学区）的简化机制，论证该机制能与基于个体风险评分的机制同样高效地定位干预对象。即便个体预测高度精准且存在有效干预措施，该结论依然成立。除提出这一简化机制外，本研究还为教育研究者关于辍学结构性决定因素的共识提供了新颖的实证支撑。综合来看，当结果受高度不平等驱动时，我们的发现对个体预测的边际价值提出了质疑。