Learning from the collective knowledge of data dispersed across private sources can provide neural networks with enhanced generalization capabilities. Federated learning, a method for collaboratively training a machine learning model across remote clients, achieves this by combining client models via the orchestration of a central server. However, current approaches face two critical limitations: i) they struggle to converge when client domains are sufficiently different, and ii) current aggregation techniques produce an identical global model for each client. In this work, we address these issues by reformulating the typical federated learning setup: rather than learning a single global model, we learn N models each optimized for a common objective. To achieve this, we apply a weighted distance minimization to model parameters shared in a peer-to-peer topology. The resulting framework, Iterative Parameter Alignment, applies naturally to the cross-silo setting, and has the following properties: (i) a unique solution for each participant, with the option to globally converge each model in the federation, and (ii) an optional early-stopping mechanism to elicit fairness among peers in collaborative learning settings. These characteristics jointly provide a flexible new framework for iteratively learning from peer models trained on disparate datasets. We find that the technique achieves competitive results on a variety of data partitions compared to state-of-the-art approaches. Further, we show that the method is robust to divergent domains (i.e. disjoint classes across peers) where existing approaches struggle.

翻译：从分散于私有数据源中的集体知识进行学习，可增强神经网络的泛化能力。联邦学习作为一种跨远程客户端协作训练机器学习模型的方法，通过中央服务器协调并聚合客户端模型来实现这一目标。然而，现有方法面临两个关键限制：i) 当客户端领域存在显著差异时，模型难以收敛；ii) 当前聚合技术为每个客户端生成完全相同的全局模型。针对这些问题，本文通过重构典型联邦学习框架提出解决方案：不再学习单一全局模型，而是学习N个针对共同目标分别优化后的模型。为此，我们在点对点拓扑中共享的模型参数上应用加权距离最小化策略。由此产生的框架——迭代参数对齐，天然适用于跨筒仓场景，并具备以下特性：(i) 每个参与者获得唯一解，同时可选择性实现联邦中各模型的全局收敛；(ii) 引入可选早停机制，以促进协作学习场景中的同伴公平性。这些特性共同构建了一个灵活的新型框架，能够对基于不同数据集训练的同伴模型进行迭代学习。实验表明，该技术在多种数据划分场景下均取得与最先进方法相匹敌的结果。尤其值得指出的是，该方法对现有方法难以处理的异构域（即客户端间存在完全不相交的类别分布）具有鲁棒性。