Federated learning has gained significant attention due to its groundbreaking ability to enable distributed learning while maintaining privacy constraints. However, as a consequence of data heterogeneity among decentralized devices, it inherently experiences significant learning degradation and slow convergence speed. Therefore, it is natural to employ the concept of clustering homogeneous clients into the same group, allowing only the model weights within each group to be aggregated. While most existing clustered federated learning methods employ either model gradients or inference outputs as metrics for client partitioning, with the goal of grouping similar devices together, may still have heterogeneity within each cluster. Moreover, there is a scarcity of research exploring the underlying reasons for determining the appropriate timing for clustering, resulting in the common practice of assigning each client to its own individual cluster, particularly in the context of highly non independent and identically distributed (Non-IID) data. In this paper, we introduce a two-stage decoupling federated learning algorithm with adaptive personalization layers named FedTSDP, where client clustering is performed twice according to inference outputs and model weights, respectively. Hopkins amended sampling is adopted to determine the appropriate timing for clustering and the sampling weight of public unlabeled data. In addition, a simple yet effective approach is developed to adaptively adjust the personalization layers based on varying degrees of data skew. Experimental results show that our proposed method has reliable performance on both IID and non-IID scenarios.

翻译：联邦学习因其在保持隐私约束的同时实现分布式学习的突破性能力而受到广泛关注。然而，由于去中心化设备间的数据异质性，联邦学习天然面临显著的学习性能下降和收敛速度缓慢问题。因此，将同质客户端聚类到同一组、仅允许组内模型权重聚合的思路自然可行。现有大多数聚类联邦学习方法采用模型梯度或推理输出作为客户端划分指标，旨在将相似设备归为一组，但各聚类内部仍可能存在异质性。此外，针对确定聚类适当时机的内在原因研究较为匮乏，导致在高度非独立同分布（Non-IID）数据场景下，常出现将每个客户端单独分配至独立聚类的做法。本文提出一种具有自适应个性化层的两阶段解耦联邦学习算法FedTSDP，该算法分别依据推理输出与模型权重进行两次客户端聚类。采用Hopkins修正采样确定聚类适当时机及公开无标注数据的采样权重。同时，开发了一种简单有效的方法，可根据数据偏斜程度自适应调整个性化层。实验结果表明，所提方法在IID与非IID场景下均具有可靠性能。