Federated learning is an important framework in modern machine learning that seeks to integrate the training of learning models from multiple users, each user having their own local data set, in a way that is sensitive to data privacy and to communication loss constraints. In clustered federated learning, one assumes an additional unknown group structure among users, and the goal is to train models that are useful for each group, rather than simply training a single global model for all users. In this paper, we propose a novel solution to the problem of clustered federated learning that is inspired by ideas in consensus-based optimization (CBO). Our new CBO-type method is based on a system of interacting particles that is oblivious to group memberships. Our model is motivated by rigorous mathematical reasoning, including a mean field analysis describing the large number of particles limit of our particle system, as well as convergence guarantees for the simultaneous global optimization of general non-convex objective functions (corresponding to the loss functions of each cluster of users) in the mean-field regime. Experimental results demonstrate the efficacy of our FedCBO algorithm compared to other state-of-the-art methods and help validate our methodological and theoretical work.

翻译：联邦学习是当代机器学习中的重要框架，旨在整合多用户（每位用户拥有各自本地数据集）的模型训练过程，同时兼顾数据隐私保护与通信损耗约束。在聚簇联邦学习中，假设用户间存在未知的群组结构，其目标是为每个群组训练有用模型，而非简单地为所有用户训练单一全局模型。本文受共识优化（CBO）思想启发，提出一种解决聚簇联邦学习问题的新方法。该新型CBO类方法基于一个对群组归属无感知的粒子交互系统，其模型构建依托严谨数学推导：包含描述粒子系统在大规模粒子极限下的平均场分析，以及均值场框架下通用非凸目标函数（对应各用户簇的损失函数）同步全局优化的收敛性保证。实验结果表明，本算法相较于其他前沿方法具有优越性，有效验证了方法论与理论工作的有效性。