When training data are fragmented across batches or federated-learned across different geographic locations, trained models manifest performance degradation. That degradation partly owes to covariate shift induced by data having been fragmented across time and space and producing dissimilar empirical training distributions. Each fragment's distribution is slightly different to a hypothetical unfragmented training distribution of covariates, and to the single validation distribution. To address this problem, we propose Fisher Information for Robust fEderated validation (\textbf{FIRE}). This method accumulates fragmentation-induced covariate shift divergences from the global training distribution via an approximate Fisher information. That term, which we prove to be a more computationally-tractable estimate, is then used as a per-fragment loss penalty, enabling scalable distribution alignment. FIRE outperforms importance weighting benchmarks by $5.1\%$ at maximum and federated learning (FL) benchmarks by up to $5.3\%$ on shifted validation sets.

翻译：当训练数据在不同批次间分散或在不同的地理位置进行联邦学习时，训练出的模型会表现出性能下降。这种性能下降部分归因于数据在时间和空间上的分散所导致的协变量偏移，从而产生了不同的经验训练分布。每个数据片段的分布与假设的未分散协变量训练分布以及单一的验证分布都存在细微差异。为解决此问题，我们提出了用于鲁棒联邦验证的费舍尔信息量（\textbf{FIRE}）。该方法通过近似的费舍尔信息量，累积了由数据分散引起的、相对于全局训练分布的协变量偏移散度。我们证明该估计量在计算上更具可操作性，随后将其用作每个数据片段的损失惩罚项，从而实现可扩展的分布对齐。在发生偏移的验证集上，FIRE 方法在重要性加权基准上的最大性能提升为 $5.1\%$，在联邦学习（FL）基准上的最大性能提升为 $5.3\%$。