Federated learning (FL) enhances data privacy with collaborative in-situ training on decentralized clients. Nevertheless, FL encounters challenges due to non-independent and identically distributed (non-i.i.d) data, leading to potential performance degradation and hindered convergence. While prior studies predominantly addressed the issue of skewed label distribution, our research addresses a crucial yet frequently overlooked problem known as multi-domain FL. In this scenario, clients' data originate from diverse domains with distinct feature distributions, instead of label distributions. To address the multi-domain problem in FL, we propose a novel method called Federated learning Without normalizations (FedWon). FedWon draws inspiration from the observation that batch normalization (BN) faces challenges in effectively modeling the statistics of multiple domains, while existing normalization techniques possess their own limitations. In order to address these issues, FedWon eliminates the normalization layers in FL and reparameterizes convolution layers with scaled weight standardization. Through extensive experimentation on five datasets and five models, our comprehensive experimental results demonstrate that FedWon surpasses both FedAvg and the current state-of-the-art method (FedBN) across all experimental setups, achieving notable accuracy improvements of more than 10% in certain domains. Furthermore, FedWon is versatile for both cross-silo and cross-device FL, exhibiting robust domain generalization capability, showcasing strong performance even with a batch size as small as 1, thereby catering to resource-constrained devices. Additionally, FedWon can also effectively tackle the challenge of skewed label distribution.

翻译：联邦学习（FL）通过分散客户端的协作原位训练增强数据隐私性。然而，由于非独立同分布（non-i.i.d）数据的存在，FL面临性能下降和收敛受阻的挑战。尽管先前研究主要关注标签分布倾斜问题，但本研究聚焦于一个关键却常被忽视的问题——多域联邦学习（multi-domain FL）。在此场景中，客户端数据来自具有不同特征分布的多个领域，而非标签分布差异。为解决FL中的多域问题，我们提出一种名为“无归一化联邦学习（FedWon）”的新方法。FedWon的灵感来源于以下观察：批量归一化（BN）难以有效建模多域的统计特征，而现有归一化技术存在各自局限性。针对这些问题，FedWon移除FL中的归一化层，并使用缩放权重标准化重建卷积层。通过在五个数据集和五个模型上的广泛实验，我们的综合结果表明：在所有实验设置中，FedWon均优于FedAvg及当前最先进方法（FedBN），在特定领域实现了超过10%的显著准确率提升。此外，FedWon兼具跨孤岛与跨设备FL的通用性，展现出强大的域泛化能力，即便在批量大小仅为1时仍表现优异，从而适用于资源受限设备。最后，FedWon还能有效应对标签分布倾斜的挑战。