Federated learning has received great attention for its capability to train a large-scale model in a decentralized manner without needing to access user data directly. It helps protect the users' private data from centralized collecting. Unlike distributed machine learning, federated learning aims to tackle non-IID data from heterogeneous sources in various real-world applications, such as those on smartphones. Existing federated learning approaches usually adopt a single global model to capture the shared knowledge of all users by aggregating their gradients, regardless of the discrepancy between their data distributions. However, due to the diverse nature of user behaviors, assigning users' gradients to different global models (i.e., centers) can better capture the heterogeneity of data distributions across users. Our paper proposes a novel multi-center aggregation mechanism for federated learning, which learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. We formulate the problem as a joint optimization that can be efficiently solved by a stochastic expectation maximization (EM) algorithm. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.

翻译：联邦学习因其在无需直接获取用户数据的情况下以分散方式培训大规模模型的能力而得到极大关注。它有助于保护用户的私人数据不被集中收集。与分布式机器学习不同,联邦学习的目的是解决各种现实世界应用,如智能手机应用中不同来源的非IID数据问题。现有的联邦学习方法通常采用单一的全球模型,通过汇总所有用户的梯度来获取共享知识,而不论其数据分布差异如何。然而,由于用户行为的多样性,将用户的梯度分配给不同的全球模型(即中心)可以更好地捕捉到用户之间数据分布的异质性。我们的论文建议采用新的多中心汇总机制进行联邦化学习,从非IID用户数据中学习多个全球模型,同时获得用户和中心之间的最佳匹配。我们把这个问题当作一种联合优化,通过随机预期最大化(EM)的算法可以有效解决。我们在基准数据集上的实验结果显示,我们的方法超越了几种大众联合学习方法。