When predicting future events, it is common to issue forecasts that are probabilistic, in the form of probability distributions over the range of possible outcomes. Such forecasts can be evaluated using proper scoring rules. Proper scoring rules condense forecast performance into a single numerical value, allowing competing forecasters to be ranked and compared. To facilitate the use of scoring rules in practical applications, the scoringRules package in R provides popular scoring rules for a wide range of forecast distributions. This paper discusses an extension to the scoringRules package that additionally permits the implementation of popular weighted scoring rules. Weighted scoring rules allow particular outcomes to be targeted during forecast evaluation, recognising that certain outcomes are often of more interest than others when assessing forecast quality. This introduces the potential for very flexible, user-oriented evaluation of probabilistic forecasts. We discuss the theory underlying weighted scoring rules, and describe how they can readily be implemented in practice using scoringRules. Functionality is available for weighted versions of several popular scoring rules, including the logarithmic score, the continuous ranked probability score (CRPS), and the energy score. Two case studies are presented to demonstrate this, whereby weighted scoring rules are applied to univariate and multivariate probabilistic forecasts in the fields of meteorology and economics.
翻译:预测未来事件时,通常以概率形式发布预测,即针对所有可能结果范围的概率分布。此类预测可通过适当评分规则进行评估。适当评分规则将预测性能浓缩为单一数值,便于对竞争性预测者进行排序与比较。为促进评分规则在实际应用中的使用,R语言中的scoringRules软件包为多种预测分布提供了常用评分规则。本文讨论该软件包的扩展功能,新增支持实施常用加权评分规则。加权评分规则允许在评估过程中针对特定结果进行加权,认识到在评判预测质量时,某些结果往往比其他结果更受关注。这为实现高度灵活、面向用户的概率预测评估提供了可能。我们讨论了加权评分规则的理论基础,并描述了如何利用scoringRules软件包在实际中便捷实现这些规则。该软件包现支持对数得分、连续排序概率得分(CRPS)及能量得分等多种常用评分规则的加权版本。我们通过两个案例研究进行演示,分别将加权评分规则应用于气象学与经济学领域的单变量及多变量概率预测。