Federated learning has received great attention for its capability to train a large-scale model in a decentralized manner without needing to access user data directly. It helps protect the users' private data from centralized collecting. Unlike distributed machine learning, federated learning aims to tackle non-IID data from heterogeneous sources in various real-world applications, such as those on smartphones. Existing federated learning approaches usually adopt a single global model to capture the shared knowledge of all users by aggregating their gradients, regardless of the discrepancy between their data distributions. However, due to the diverse nature of user behaviors, assigning users' gradients to different global models (i.e., centers) can better capture the heterogeneity of data distributions across users. Our paper proposes a novel multi-center aggregation mechanism for federated learning, which learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. We formulate the problem as a joint optimization that can be efficiently solved by a stochastic expectation maximization (EM) algorithm. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.

翻译：联邦学习因其无需直接访问用户数据即可以去中心化方式训练大规模模型的能力而备受关注，有助于保护用户隐私数据免于集中收集。与分布式机器学习不同，联邦学习旨在处理各类真实应用（如智能手机应用）中异构来源的非独立同分布（non-IID）数据。现有联邦学习方案通常采用单一全局模型，通过聚合所有用户的梯度来捕获其共享知识，而忽略不同用户数据分布的差异。然而，鉴于用户行为的多样性，将用户梯度分配至不同的全局模型（即中心）能更有效地捕捉用户间数据分布的异质性。本文提出一种面向联邦学习的创新多中心聚合机制，该机制从非IID用户数据中学习多个全局模型，并同时推导用户与中心之间的最优匹配。我们将该问题建模为联合优化问题，可通过随机期望最大化（EM）算法高效求解。在基准数据集上的实验结果表明，我们的方法优于多种主流联邦学习方案。