Federated Learning (FL) is a distributed machine learning paradigm that achieves a globally robust model through decentralized computation and periodic model synthesis, primarily focusing on the global model's accuracy over aggregated datasets of all participating clients. Personalized Federated Learning (PFL) instead tailors exclusive models for each client, aiming to enhance the accuracy of clients' individual models on specific local data distributions. Despite of their wide adoption, existing FL and PFL works have yet to comprehensively address the class-imbalance issue, one of the most critical challenges within the realm of data heterogeneity in PFL and FL research. In this paper, we propose FedReMa, an efficient PFL algorithm that can tackle class-imbalance by 1) utilizing an adaptive inter-client co-learning approach to identify and harness different clients' expertise on different data classes throughout various phases of the training process, and 2) employing distinct aggregation methods for clients' feature extractors and classifiers, with the choices informed by the different roles and implications of these model components. Specifically, driven by our experimental findings on inter-client similarity dynamics, we develop critical co-learning period (CCP), wherein we introduce a module named maximum difference segmentation (MDS) to assess and manage task relevance by analyzing the similarities between clients' logits of their classifiers. Outside the CCP, we employ an additional scheme for model aggregation that utilizes historical records of each client's most relevant peers to further enhance the personalization stability. We demonstrate the superiority of our FedReMa in extensive experiments.

翻译：联邦学习（FL）是一种分布式机器学习范式，通过去中心化计算和周期性模型聚合实现全局鲁棒模型，主要关注全局模型在所有参与客户端聚合数据集上的准确性。个性化联邦学习（PFL）则旨在为每个客户端定制专属模型，以提升客户端个体模型在特定本地数据分布上的准确性。尽管应用广泛，现有FL与PFL研究尚未能全面解决类别不平衡问题——这是PFL与FL研究中数据异质性领域最关键的挑战之一。本文提出FedReMa，一种高效的PFL算法，可通过以下方式应对类别不平衡：1）采用自适应客户端间协同学习策略，在训练过程的不同阶段识别并利用不同客户端在不同数据类别上的专业能力；2）对客户端的特征提取器和分类器采用差异化聚合方法，其选择依据这些模型组件的不同作用与影响。具体而言，基于我们对客户端间相似性动态的实验发现，我们提出了关键协同学习周期（CCP），在其中引入名为最大差异分割（MDS）的模块，通过分析客户端分类器逻辑值的相似性来评估和管理任务相关性。在CCP之外，我们采用额外的模型聚合方案，该方案利用每个客户端最相关协作方的历史记录，以进一步增强个性化稳定性。我们通过大量实验证明了FedReMa的优越性。