Ensemble forecasts often outperform forecasts from individual standalone models, and have been used to support decision-making and policy planning in various fields. As collaborative forecasting efforts to create effective ensembles grow, so does interest in understanding individual models' relative importance in the ensemble. To this end, we propose two practical methods that measure the difference between ensemble performance when a given model is or is not included in the ensemble: a leave-one-model-out algorithm and a leave-all-subsets-of-models-out algorithm, which is based on the Shapley value. We explore the relationship between these metrics, forecast accuracy, and the similarity of errors, both analytically and through simulations. We illustrate this measure of the value a component model adds to an ensemble in the presence of other models using US COVID-19 death forecasts. This study offers valuable insight into individual models' unique features within an ensemble, which standard accuracy metrics alone cannot reveal.

翻译：集成预测通常优于单个独立模型的预测，并已被用于支持各领域的决策制定与政策规划。随着创建有效集成模型的协作预测工作日益增多，理解个体模型在集成中的相对重要性也愈发受到关注。为此，我们提出了两种实用方法，用于衡量给定模型是否包含在集成中时集成性能的差异：一种是留一模型法，另一种是基于沙普利值的留所有模型子集法。我们通过理论分析和模拟实验，探讨了这些度量指标与预测准确性及误差相似性之间的关系。我们以美国COVID-19死亡预测为例，说明了在存在其他模型的情况下，组件模型为集成所增添的价值度量。本研究为理解个体模型在集成中的独特特征提供了有价值的见解，这是单纯依靠标准准确度指标所无法揭示的。