Federated learning has gained significant attention due to its groundbreaking ability to enable distributed learning while maintaining privacy constraints. However, as a consequence of data heterogeneity among decentralized devices, it inherently experiences significant learning degradation and slow convergence speed. Therefore, it is natural to employ the concept of clustering homogeneous clients into the same group, allowing only the model weights within each group to be aggregated. While most existing clustered federated learning methods employ either model gradients or inference outputs as metrics for client partitioning, with the goal of grouping similar devices together, may still have heterogeneity within each cluster. Moreover, there is a scarcity of research exploring the underlying reasons for determining the appropriate timing for clustering, resulting in the common practice of assigning each client to its own individual cluster, particularly in the context of highly non independent and identically distributed (Non-IID) data. In this paper, we introduce a two-stage decoupling federated learning algorithm with adaptive personalization layers named FedTSDP, where client clustering is performed twice according to inference outputs and model weights, respectively. Hopkins amended sampling is adopted to determine the appropriate timing for clustering and the sampling weight of public unlabeled data. In addition, a simple yet effective approach is developed to adaptively adjust the personalization layers based on varying degrees of data skew. Experimental results show that our proposed method has reliable performance on both IID and non-IID scenarios.

翻译：联邦学习因其在维护隐私约束的同时实现分布式学习的突破性能力而受到广泛关注。然而，由于分散设备间的数据异质性，联邦学习本质上存在显著的学习性能下降和收敛速度缓慢的问题。因此，自然可采用将同质客户端聚类至同一组的概念，仅允许组内模型权重进行聚合。尽管现有大多数聚类联邦学习方法采用模型梯度或推理输出作为客户端划分的度量指标，旨在将相似设备归为一组，但各聚类内部仍可能存在异质性。此外，目前缺乏探索确定聚类适当时机根本原因的研究，导致常见做法是将每个客户端分配至独立聚类——尤其在高度非独立同分布（Non-IID）数据场景下。本文提出一种具有自适应个性化层的两阶段解耦联邦学习算法FedTSDP，该算法分别依据推理输出和模型权重执行两次客户端聚类。采用霍普金斯修正采样确定聚类适当时机及公共无标签数据的采样权重。此外，我们开发了一种简洁而有效的方法，可根据数据偏斜程度自适应调整个性化层。实验结果表明，所提方法在独立同分布（IID）与非独立同分布场景下均具有可靠性能。