Federated Learning (FL) has emerged as a promising distributed learning paradigm that enables multiple clients to learn a global model collaboratively without sharing their private data. However, the effectiveness of FL is highly dependent on the quality of the data that is being used for training. In particular, data heterogeneity issues, such as label distribution skew and feature skew, can significantly impact the performance of FL. Previous studies in FL have primarily focused on addressing label distribution skew data heterogeneity, while only a few recent works have made initial progress in tackling feature skew issues. Notably, these two forms of data heterogeneity have been studied separately and have not been well explored within a unified FL framework. To address this gap, we propose Fed-CO$_{2}$, a universal FL framework that handles both label distribution skew and feature skew within a \textbf{C}ooperation mechanism between the \textbf{O}nline and \textbf{O}ffline models. Specifically, the online model learns general knowledge that is shared among all clients, while the offline model is trained locally to learn the specialized knowledge of each individual client. To further enhance model cooperation in the presence of feature shifts, we design an intra-client knowledge transfer mechanism that reinforces mutual learning between the online and offline models, and an inter-client knowledge transfer mechanism to increase the models' domain generalization ability. Extensive experiments show that our Fed-CO$_{2}$ outperforms a wide range of existing personalized federated learning algorithms in terms of handling label distribution skew and feature skew, both individually and collectively. The empirical results are supported by our convergence analyses in a simplified setting.

翻译：联邦学习（FL）作为一种有前景的分布式学习范式涌现，它允许多个客户端在不共享私有数据的情况下协作学习全局模型。然而，FL的有效性高度依赖于用于训练的数据质量。特别是标签分布偏移和特征偏移等数据异质性问题，会显著影响FL的性能。以往FL研究主要侧重于解决标签分布偏移型数据异质性，仅少数近期工作在解决特征偏移问题上取得初步进展。值得注意的是，这两种数据异质性形式一直被分开研究，尚未在统一的FL框架中得到充分探索。为弥补这一空白，我们提出Fed-CO$_{2}$，一个通用的FL框架，通过**在**线与**离**线模型之间的**协作**机制同时处理标签分布偏移和特征偏移。具体而言，在线模型学习所有客户端共享的通用知识，而离线模型则在本地训练以学习每个客户端特定的专业知识。为进一步增强存在特征偏移时的模型协作，我们设计了客户端内知识迁移机制以强化在线与离线模型之间的相互学习，以及客户端间知识迁移机制以提升模型的领域泛化能力。大量实验表明，我们的Fed-CO$_{2}$在处理标签分布偏移和特征偏移（无论是单独还是联合情况）方面，优于现有多种个性化联邦学习算法。实证结果得到了简化设置下收敛性分析的支持。