Federated learning (FL) enhances data privacy with collaborative in-situ training on decentralized clients. Nevertheless, FL encounters challenges due to non-independent and identically distributed (non-i.i.d) data, leading to potential performance degradation and hindered convergence. While prior studies predominantly addressed the issue of skewed label distribution, our research addresses a crucial yet frequently overlooked problem known as multi-domain FL. In this scenario, clients' data originate from diverse domains with distinct feature distributions, as opposed to label distributions. To address the multi-domain problem in FL, we propose a novel method called Federated learning Without normalizations (FedWon). FedWon draws inspiration from the observation that batch normalization (BN) faces challenges in effectively modeling the statistics of multiple domains, while alternative normalization techniques possess their own limitations. In order to address these issues, FedWon eliminates all normalizations in FL and reparameterizes convolution layers with scaled weight standardization. Through comprehensive experimentation on four datasets and four models, our results demonstrate that FedWon surpasses both FedAvg and the current state-of-the-art method (FedBN) across all experimental setups, achieving notable improvements of over 10% in certain domains. Furthermore, FedWon is versatile for both cross-silo and cross-device FL, exhibiting strong performance even with a batch size as small as 1, thereby catering to resource-constrained devices. Additionally, FedWon effectively tackles the challenge of skewed label distribution.

翻译：联邦学习通过客户端本地协同训练增强了数据隐私保护。然而，由于数据非独立同分布特性，联邦学习面临性能下降和收敛困难的挑战。现有研究主要关注标签分布偏移问题，而本文聚焦于一个关键却常被忽视的多领域联邦学习场景——即客户端数据来自具有不同特征分布而非标签分布的多个领域。为解决联邦学习中的多领域问题，我们提出联邦无归一化方法。该方法源于以下发现：批归一化难以有效建模多领域的统计特征，而其他归一化技术各有局限。为此，FedWon消除联邦学习中的所有归一化操作，通过缩放权重标准化对卷积层进行参数重定义。在四个数据集和四个模型上的综合实验表明，FedWon在所有实验设置中均超越FedAvg和当前最先进方法FedBN，某些领域性能提升超过10%。此外，FedWon同时适用于跨孤岛和跨设备联邦学习，即使批大小仅为1时仍表现优异，可适配资源受限设备。实验还证实FedWon能有效应对标签分布偏移挑战。