Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training data locally. Unlike the centralized training setting, the non-IID and imbalanced (statistical heterogeneity) training data of FL is distributed in the federated network, which will increase the divergences between the local models and global model and further degrade the performance. In this paper, we propose a novel federated learning framework FedGroup based on a similarity-based clustering strategy, in which we 1) group the training of clients based on the ternary cosine similarities between the clients' parameter updates; 2) reduce the clustering complexity of high-dimension low-sample size (HDLSS) parameter updates data by decomposed the direction vectors to derive the ternary cosine similarity metrics for clustering. FedGroup can achieve improvements by dividing joint optimization into groups of sub-optimization, and can be combined with \textit{FedProx}, the state-of-the-art federated optimization algorithm. We evaluate FedGroup and FedGrouProx (combined with FedProx) on several open datasets. The experimental results show that our proposed frameworks significantly improving absolute test accuracy by +14.7% on FEMNIST compared to FedAvg, +5.4% on Sentiment140 compared to FedProx.

翻译：联邦学习联合会(FL) 使多个参与设备能够合作为全球神经网络模式作出贡献,同时保持当地的培训数据。 与中央培训环境不同,FL的非IID和不平衡(统计异质性)培训数据在联邦网络中分布,这将增加本地模型和全球模型之间的差异,并进一步降低绩效。 在本文件中,我们提议基于类似基于集群战略的FedGroup 创新的联邦学习框架FedGroup(我们1) 将客户培训基于客户参数更新之间长期共生异性;2 降低高二分低抽样规模(HDLSS)参数的组合复杂性,通过解析方向矢量来得出用于集群的永久共生性相似度度度指标来更新数据。 FedGroup可以将联合优化分为次优化组,并可以与\textit{Fedprox}、 州-艺术联合优化数种异性调整算法(我们通过FedGroGroup 和FedGEMx的绝对值测试框架对FFDGroup 做了大幅的对比。