Learning from the collective knowledge of data dispersed across private sources can provide neural networks with enhanced generalization capabilities. Federated learning, a method for collaboratively training a machine learning model across remote clients, achieves this by combining client models via the orchestration of a central server. However, current approaches face two critical limitations: i) they struggle to converge when client domains are sufficiently different, and ii) current aggregation techniques produce an identical global model for each client. In this work, we address these issues by reformulating the typical federated learning setup: rather than learning a single global model, we learn N models each optimized for a common objective. To achieve this, we apply a weighted distance minimization to model parameters shared in a peer-to-peer topology. The resulting framework, Iterative Parameter Alignment, applies naturally to the cross-silo setting, and has the following properties: (i) a unique solution for each participant, with the option to globally converge each model in the federation, and (ii) an optional early-stopping mechanism to elicit fairness among peers in collaborative learning settings. These characteristics jointly provide a flexible new framework for iteratively learning from peer models trained on disparate datasets. We find that the technique achieves competitive results on a variety of data partitions compared to state-of-the-art approaches. Further, we show that the method is robust to divergent domains (i.e. disjoint classes across peers) where existing approaches struggle.

翻译：从分散在私有来源中的数据集体知识中学习，能够增强神经网络的泛化能力。联邦学习作为一种跨远程客户端协同训练机器学习模型的方法，通过中央服务器协调客户端模型来实现这一目标。然而，现有方法面临两个关键局限：i) 当客户端领域差异较大时难以收敛，ii) 当前聚合技术为每个客户端生成完全相同的全局模型。为解决这些问题，本文重新定义了典型联邦学习框架：不再学习单一全局模型，而是学习N个各自针对共同目标优化的模型。为此，我们采用加权距离最小化方法处理点对点拓扑中共享的模型参数。所提出的迭代参数对齐框架自然地适用于跨孤立数据场景，具备以下特性：(i) 为每个参与者生成唯一解，并可选实现联邦内各模型的全局收敛；(ii) 引入可选早停机制以在协作学习场景中促进同伴间的公平性。这些特性共同构建了一个灵活的新框架，支持从基于异构数据集训练的同行模型中迭代学习。实验表明，与现有最优方法相比，该技术在各种数据划分上均取得具有竞争力的结果。此外，我们证明该方法在现有方法难以处理的孤立领域差异（即跨客户端的不相交类别）场景中具有鲁棒性。