Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative training of machine learning models over decentralized devices without exposing their local data. One of the major challenges in FL is the presence of uneven data distributions across client devices, violating the well-known assumption of independent-and-identically-distributed (IID) training samples in conventional machine learning. To address the performance degradation issue incurred by such data heterogeneity, clustered federated learning (CFL) shows its promise by grouping clients into separate learning clusters based on the similarity of their local data distributions. However, state-of-the-art CFL approaches require a large number of communication rounds to learn the distribution similarities during training until the formation of clusters is stabilized. Moreover, some of these algorithms heavily rely on a predefined number of clusters, thus limiting their flexibility and adaptability. In this paper, we propose {\em FedClust}, a novel approach for CFL that leverages the correlation between local model weights and the data distribution of clients. {\em FedClust} groups clients into clusters in a one-shot manner by measuring the similarity degrees among clients based on the strategically selected partial weights of locally trained models. We conduct extensive experiments on four benchmark datasets with different non-IID data settings. Experimental results demonstrate that {\em FedClust} achieves higher model accuracy up to $\sim$45\% as well as faster convergence with a significantly reduced communication cost up to 2.7$\times$ compared to its state-of-the-art counterparts.

翻译：联邦学习（Federated Learning, FL）是一种新兴的分布式机器学习范式，它使得去中心化设备能够在不暴露本地数据的情况下协作训练机器学习模型。联邦学习面临的主要挑战之一是客户端设备间存在不均匀的数据分布，这违背了传统机器学习中训练样本独立同分布（IID）的经典假设。为应对此类数据异构性导致的性能下降问题，聚类联邦学习（Clustered Federated Learning, CFL）展现出其潜力，它通过根据客户端本地数据分布的相似性将其分组到不同的学习聚类中。然而，现有的先进CFL方法需要大量通信轮次来学习训练过程中的分布相似性，直至聚类结构稳定。此外，其中一些算法严重依赖于预定义的聚类数量，从而限制了其灵活性与适应性。本文提出一种新颖的CFL方法——{\em FedClust}，该方法利用本地模型权重与客户端数据分布之间的相关性。{\em FedClust}通过基于策略性选取的本地训练模型部分权重来衡量客户端间的相似度，从而以一次性方式将客户端分组为聚类。我们在四个基准数据集上使用不同的非IID数据设置进行了大量实验。实验结果表明，与现有先进方法相比，{\em FedClust}实现了高达$\sim$45\%的模型精度提升，并以显著降低高达2.7$\times$的通信成本实现了更快的收敛速度。