Despite the significance of probabilistic time-series forecasting models, their evaluation metrics often involve intractable integrations. The most widely used metric, the continuous ranked probability score (CRPS), is a strictly proper scoring function; however, its computation requires approximation. We found that popular CRPS estimators--specifically, the quantile-based estimator implemented in the widely used GluonTS library and the probability-weighted moment approximation--both exhibit inherent estimation biases. These biases lead to crude approximations, resulting in improper rankings of forecasting model performance when CRPS values are close. To address this issue, we introduced a kernel quadrature approach that leverages an unbiased CRPS estimator and employs cubature construction for scalable computation. Empirically, our approach consistently outperforms the two widely used CRPS estimators.

翻译：尽管概率时间序列预测模型具有重要意义，但其评估指标常涉及难以处理的积分。最广泛使用的指标——连续排序概率评分（CRPS）是一种严格恰当评分函数；然而，其计算需要近似处理。我们发现流行的CRPS估计器——特别是广泛使用的GluonTS库中实现的分位数估计器以及概率加权矩近似法——均存在固有的估计偏差。这些偏差会导致粗糙的近似结果，当CRPS值接近时，将导致预测模型性能的排序失当。为解决此问题，我们引入了一种核求积方法，该方法利用无偏CRPS估计器，并采用体积分构造实现可扩展计算。实验表明，我们的方法在实证中持续优于两种广泛使用的CRPS估计器。