Recent years have seen increasing efforts to forecast infectious disease burdens, with a primary goal being to help public health workers make informed policy decisions. However, there has only been limited discussion of how predominant forecast evaluation metrics might indicate the success of policies based in part on those forecasts. We explore one possible tether between forecasts and policy: the allocation of limited medical resources so as to minimize unmet need. We use probabilistic forecasts of disease burden in each of several regions to determine optimal resource allocations, and then we score forecasts according to how much unmet need their associated allocations would have allowed. We illustrate with forecasts of COVID-19 hospitalizations in the US, and we find that the forecast skill ranking given by this allocation scoring rule can vary substantially from the ranking given by the weighted interval score. We see this as evidence that the allocation scoring rule detects forecast value that is missed by traditional accuracy measures and that the general strategy of designing scoring rules that are directly linked to policy performance is a promising direction for epidemic forecast evaluation.

翻译：近年来，人们日益重视对传染病负担的预测工作，其主要目标是帮助公共卫生工作者制定明智的政策决策。然而，关于主流预测评估指标如何反映基于这些预测的政策成效，相关讨论仍十分有限。本文探索了预测与政策之间的一种潜在纽带：通过分配有限的医疗资源以最大程度减少未满足需求。我们利用多个区域疾病负担的概率预测来确定最优资源分配方案，然后根据相关分配所能避免的未满足需求程度对预测进行评分。我们以美国新冠肺炎住院率预测为例进行说明，发现该分配评分规则给出的预测能力排序与加权区间评分给出的排序存在显著差异。我们认为这证明分配评分规则能捕捉传统准确性指标遗漏的预测价值，而设计直接关联政策成效的评分规则这一总体策略，是传染病预测评估领域颇具前景的发展方向。