Federated Learning (FL) has emerged as a promising distributed learning paradigm that enables multiple clients to learn a global model collaboratively without sharing their private data. However, the effectiveness of FL is highly dependent on the quality of the data that is being used for training. In particular, data heterogeneity issues, such as label distribution skew and feature skew, can significantly impact the performance of FL. Previous studies in FL have primarily focused on addressing label distribution skew data heterogeneity, while only a few recent works have made initial progress in tackling feature skew issues. Notably, these two forms of data heterogeneity have been studied separately and have not been well explored within a unified FL framework. To address this gap, we propose Fed-CO$_{2}$, a universal FL framework that handles both label distribution skew and feature skew within a \textbf{C}ooperation mechanism between the \textbf{O}nline and \textbf{O}ffline models. Specifically, the online model learns general knowledge that is shared among all clients, while the offline model is trained locally to learn the specialized knowledge of each individual client. To further enhance model cooperation in the presence of feature shifts, we design an intra-client knowledge transfer mechanism that reinforces mutual learning between the online and offline models, and an inter-client knowledge transfer mechanism to increase the models' domain generalization ability. Extensive experiments show that our Fed-CO$_{2}$ outperforms a wide range of existing personalized federated learning algorithms in terms of handling label distribution skew and feature skew, both individually and collectively. The empirical results are supported by our convergence analyses in a simplified setting.

翻译：联邦学习（FL）作为一种有前景的分布式学习范式，允许多个客户端在不共享私有数据的情况下协同学习全局模型。然而，FL的有效性高度依赖于训练数据的质量。特别地，标签分布偏移和特征偏移等数据异构性问题会显著影响FL的性能。以往FL研究主要聚焦于解决标签分布偏移引发的数据异构性，仅少数近期工作在应对特征偏移问题上取得初步进展。值得注意的是，这两类数据异构性被独立研究，尚未在统一FL框架中得到充分探索。为此，我们提出Fed-CO₂——一种通用的FL框架，通过在线模型与离线模型的协同机制（Cooperation mechanism）同时处理标签分布偏移和特征偏移。具体而言，在线模型学习所有客户端共享的通用知识，而离线模型则通过本地训练学习每个客户端的专有知识。为在特征偏移场景下增强模型协同，我们设计了客户端内知识迁移机制以强化在线与离线模型的相互学习，以及客户端间知识迁移机制以提升模型的域泛化能力。大量实验表明，无论处理标签分布偏移、特征偏移，还是两者并存的情况，Fed-CO₂在性能上均优于多种现有个性化联邦学习算法。通过简化设置下的收敛性分析，我们进一步验证了实证结果的理论基础。