Federated Learning (FL) is a collaborative method for training models while preserving data privacy in decentralized settings. However, FL encounters challenges related to data heterogeneity, which can result in performance degradation. In our study, we observe that as data heterogeneity increases, feature representation in the FedAVG model deteriorates more significantly compared to classifier weight. Additionally, we observe that as data heterogeneity increases, the gap between higher feature norms for observed classes, obtained from local models, and feature norms of unobserved classes widens, in contrast to the behavior of classifier weight norms. This widening gap extends to encompass the feature norm disparities between local and the global models. To address these issues, we introduce Federated Averaging with Feature Normalization Update (FedFN), a straightforward learning method. We demonstrate the superior performance of FedFN through extensive experiments, even when applied to pretrained ResNet18. Subsequently, we confirm the applicability of FedFN to foundation models.

翻译：联邦学习（FL）是一种在去中心化环境下训练模型同时保护数据隐私的协作方法。然而，FL面临与数据异构性相关的挑战，可能导致性能下降。本研究中我们观察到，随着数据异构性增加，FedAVG模型中的特征表示相比分类器权重退化更为显著。此外，我们发现当数据异构性增强时，本地模型中观测类别的高特征范数与未观测类别特征范数之间的差距扩大，这与分类器权重范数的行为形成对比。这种扩大的差距进一步延伸至本地模型与全局模型之间的特征范数差异。为解决这些问题，我们提出了一种简洁的学习方法——带特征归一化更新的联邦平均算法（FedFN）。通过大量实验，我们证明了FedFN在预训练ResNet18中仍具有优越性能，并进一步验证了其对基础模型的适用性。