Learning from the collective knowledge of data dispersed across private sources can provide neural networks with enhanced generalization capabilities. Federated learning, a method for collaboratively training a machine learning model across remote clients, achieves this by combining client models via the orchestration of a central server. However, current approaches face two critical limitations: i) they struggle to converge when client domains are sufficiently different, and ii) current aggregation techniques produce an identical global model for each client. In this work, we address these issues by reformulating the typical federated learning setup: rather than learning a single global model, we learn N models each optimized for a common objective. To achieve this, we apply a weighted distance minimization to model parameters shared in a peer-to-peer topology. The resulting framework, Iterative Parameter Alignment, applies naturally to the cross-silo setting, and has the following properties: (i) a unique solution for each participant, with the option to globally converge each model in the federation, and (ii) an optional early-stopping mechanism to elicit fairness among peers in collaborative learning settings. These characteristics jointly provide a flexible new framework for iteratively learning from peer models trained on disparate datasets. We find that the technique achieves competitive results on a variety of data partitions compared to state-of-the-art approaches. Further, we show that the method is robust to divergent domains (i.e. disjoint classes across peers) where existing approaches struggle.

翻译：从分散在私有来源的数据中获取集体知识进行学习，能够提升神经网络的泛化能力。联邦学习通过中央服务器的协调聚合客户端模型，实现跨远程客户端的协作式机器学习模型训练。然而，现有方法面临两大关键局限：（i）当客户端领域差异显著时难以收敛；（ii）当前聚合技术为每个客户端生成相同的全局模型。本研究通过重新定义典型联邦学习框架来解决上述问题：并非学习单一全局模型，而是学习N个均针对共同目标优化的独立模型。为实现这一目标，我们对点对点拓扑中共享的模型参数施加加权距离最小化。由此产生的迭代参数对齐框架天然适用于跨数据孤岛场景，并具备以下特性：（i）为每个参与者生成唯一解，同时允许联邦中所有模型实现全局收敛；（ii）可选的早停机制，在协作学习场景中激发同伴间的公平性。这些特性共同构建了灵活的新框架，支持从基于不同数据集训练的同伴模型中迭代学习。实验表明，该技术在多种数据划分场景下均取得与前沿方法相当的竞争性结果。此外，我们证明了该方法在现有方法难以处理的领域差异性场景（即同伴间存在不相交类别）中具有鲁棒性。