Machine learning models frequently experience performance drops under distribution shifts. The underlying cause of such shifts may be multiple simultaneous factors such as changes in data quality, differences in specific covariate distributions, or changes in the relationship between label and features. When a model does fail during deployment, attributing performance change to these factors is critical for the model developer to identify the root cause and take mitigating actions. In this work, we introduce the problem of attributing performance differences between environments to distribution shifts in the underlying data generating mechanisms. We formulate the problem as a cooperative game where the players are distributions. We define the value of a set of distributions to be the change in model performance when only this set of distributions has changed between environments, and derive an importance weighting method for computing the value of an arbitrary set of distributions. The contribution of each distribution to the total performance change is then quantified as its Shapley value. We demonstrate the correctness and utility of our method on synthetic, semi-synthetic, and real-world case studies, showing its effectiveness in attributing performance changes to a wide range of distribution shifts.

翻译：机器学习模型在分布偏移下经常出现性能下降。此类偏移的潜在原因可能是多种因素的共同作用，例如数据质量的变化、特定协变量分布差异，或标签与特征之间关系的改变。当模型在部署过程中失效时，将这些性能变化归因于上述因素，对于模型开发者确定根本原因并采取缓解措施至关重要。本研究提出了一个新问题：将不同环境间的性能差异归因于底层数据生成机制中的分布偏移。我们将该问题形式化为一个合作博弈，其中博弈方为各分布。我们定义一组分布的价值为：当仅该组分布在环境间发生变化时模型性能的变化量，并推导出一种重要性加权方法，用于计算任意分布组的价值。每个分布对总性能变化的贡献随后由其Shapley值量化。我们通过合成、半合成及真实世界案例研究证明了该方法的正确性与实用性，展示了其在将性能变化归因于多种分布偏移方面的有效性。