Federated Learning (FL) is currently one of the most popular technologies in the field of Artificial Intelligence (AI) due to its collaborative learning and ability to preserve client privacy. However, it faces challenges such as non-identically and non-independently distributed (non-IID) and data with imbalanced labels among local clients. To address these limitations, the research community has explored various approaches such as using local model parameters, federated generative adversarial learning, and federated representation learning. In our study, we propose a novel Clustered FedStack framework based on the previously published Stacked Federated Learning (FedStack) framework. The local clients send their model predictions and output layer weights to a server, which then builds a robust global model. This global model clusters the local clients based on their output layer weights using a clustering mechanism. We adopt three clustering mechanisms, namely K-Means, Agglomerative, and Gaussian Mixture Models, into the framework and evaluate their performance. We use Bayesian Information Criterion (BIC) with the maximum likelihood function to determine the number of clusters. The Clustered FedStack models outperform baseline models with clustering mechanisms. To estimate the convergence of our proposed framework, we use Cyclical learning rates.

翻译：联邦学习（Federated Learning, FL）凭借其协作学习特性及保护客户隐私的能力，目前已成为人工智能领域最热门的技术之一。然而，该技术面临本地客户端数据非独立同分布（non-IID）及标签不均衡等挑战。为解决这些问题，研究社区探索了多种方法，例如采用局部模型参数、联邦生成对抗学习及联邦表示学习等。在本研究中，我们基于先前发表的堆叠联邦学习（Stacked Federated Learning, FedStack）框架，提出了一种新颖的聚类FedStack（Clustered FedStack）框架。本地客户端将其模型预测结果及输出层权重发送至服务器，服务器据此构建鲁棒的全局模型。该全局模型通过聚类机制，基于输出层权重对本地客户端进行聚类。我们将三种聚类机制——K-Means、凝聚聚类（Agglomerative）及高斯混合模型（Gaussian Mixture Models）——集成至框架中并评估其性能。采用基于最大似然函数的贝叶斯信息准则（Bayesian Information Criterion, BIC）确定聚类数量。实验表明，聚类FedStack模型在采用聚类机制后，性能显著优于基线模型。为估计所提框架的收敛性，我们采用周期性学习率（Cyclical learning rates）。