Probabilistic forecasts are typically obtained using state-of-the-art statistical and machine learning models, with model parameters estimated by optimizing a proper scoring rule over a set of training data. If the model class is not correctly specified, then the learned model will not necessarily issue forecasts that are calibrated. Calibrated forecasts allow users to appropriately balance risks in decision making, and it is particularly important that forecast models issue calibrated predictions for extreme events, since such outcomes often generate large socio-economic impacts. In this work, we study how the loss function used to train probabilistic forecast models can be adapted to improve the reliability of forecasts made for extreme events. We investigate loss functions based on weighted scoring rules, and additionally propose regularizing loss functions using a measure of tail miscalibration. We apply these approaches to a hierarchy of increasingly flexible forecast models for UK wind speeds, including simple parametric models, distributional regression networks, and conditional generative models. We demonstrate that state-of-the-art models do not issue calibrated forecasts for extreme wind speeds, and that the calibration of forecasts for extreme events can be improved by suitable adaptations to the loss function during model training. This introduces a trade-off between calibrated forecasts for extreme events and calibrated forecasts for more common outcomes.

翻译：概率预测通常采用最先进的统计和机器学习模型实现，通过优化训练数据集上的适当评分准则来估计模型参数。若模型类别未正确指定，则学习得到的模型未必能输出经过校准的预测。校准后的预测使用户能够在决策过程中合理平衡风险，尤其重要的是，预测模型需为极端事件输出校准预测，因为此类结果常产生巨大的社会经济影响。本研究探讨如何调整用于训练概率预测模型的损失函数，以提高极端事件预测的可靠性。我们研究了基于加权评分准则的损失函数，并进一步提出使用尾部误校准度量对损失函数进行正则化。将这些方法应用于英国风速预测中层次化递增的灵活模型体系，包括简单参数模型、分布回归网络和条件生成模型。我们证明，最先进模型未能对极端风速输出校准预测，而通过在模型训练期间适当调整损失函数，可改善极端事件预测的校准性。这将在极端事件的校准预测与常见结果的校准预测之间引入权衡。