Multivariate probabilistic verification is concerned with the evaluation of joint probability distributions of vector quantities such as a weather variable at multiple locations or a wind vector for instance. The logarithmic score is a proper score that is useful in this context. In order to apply this score to ensemble forecasts, a choice for the density is required. Here, we are interested in the specific case when the density is multivariate normal with mean and covariance given by the ensemble mean and ensemble covariance, respectively. Under the assumptions of multivariate normality and exchangeability of the ensemble members, a relationship is derived which describes how the logarithmic score depends on ensemble size. It permits to estimate the score in the limit of infinite ensemble size from a small ensemble and thus produces a fair logarithmic score for multivariate ensemble forecasts under the assumption of normality. This generalises a study from 2018 which derived the ensemble size adjustment of the logarithmic score in the univariate case. An application to medium-range forecasts examines the usefulness of the ensemble size adjustments when multivariate normality is only an approximation. Predictions of vectors consisting of several different combinations of upper air variables are considered. Logarithmic scores are calculated for these vectors using ECMWF's daily extended-range forecasts which consist of a 100-member ensemble. The probabilistic forecasts of these vectors are verified against operational ECMWF analyses in the Northern mid-latitudes in autumn 2023. Scores are computed for ensemble sizes from 8 to 100. The fair logarithmic scores of ensembles with different cardinalities are very close, in contrast to the unadjusted scores which decrease considerably with ensemble size. This provides evidence for the practical usefulness of the derived relationships.
翻译:多元概率验证关注的是向量量(例如多个站点的气象变量或风矢量)的联合概率分布评估。对数评分是一种在此背景下适用的严格评分方法。为将其应用于集合预报,需选择相应的概率密度函数。本文研究特定情形:当密度函数为多元正态分布,其均值与协方差矩阵分别由集合均值与集合协方差给出时的情况。在多元正态性与集合成员可交换性的假设下,推导出描述对数评分如何依赖于集合规模的关系式。该关系式允许通过小规模集合估计无限集合规模极限下的评分,从而在正态性假设下为多元集合预报提供公平的对数评分。此项研究推广了2018年针对单变量情形下对数评分的集合规模调整方法。通过对中期预报的应用,考察了当多元正态性仅为近似假设时集合规模调整方法的实用性。研究考虑了由多种高空变量不同组合构成的向量预报。利用ECMWF每日延伸期预报(包含100个成员的集合)计算这些向量的对数评分。在2023年秋季北半球中纬度地区,将这些向量的概率预报与ECMWF业务分析场进行验证。针对8至100的集合规模计算评分。调整后的公平对数评分在不同集合规模间高度接近,而未调整评分则随集合规模显著下降。这为推导关系的实际应用价值提供了证据。