In the realm of real-world devices, centralized servers in Federated Learning (FL) present challenges including communication bottlenecks and susceptibility to a single point of failure. Additionally, contemporary devices inherently exhibit model and data heterogeneity. Existing work lacks a Decentralized FL (DFL) framework capable of accommodating such heterogeneity without imposing architectural restrictions or assuming the availability of public data. To address these issues, we propose a Decentralized Federated Mutual Learning (DFML) framework that is serverless, supports nonrestrictive heterogeneous models, and avoids reliance on public data. DFML effectively handles model and data heterogeneity through mutual learning, which distills knowledge between clients, and cyclically varying the amount of supervision and distillation signals. Extensive experimental results demonstrate consistent effectiveness of DFML in both convergence speed and global accuracy, outperforming prevalent baselines under various conditions. For example, with the CIFAR-100 dataset and 50 clients, DFML achieves a substantial increase of +17.20% and +19.95% in global accuracy under Independent and Identically Distributed (IID) and non-IID data shifts, respectively.

翻译：在现实设备环境中，联邦学习（FL）中的中心化服务器存在通信瓶颈和单点故障风险等挑战。此外，当前设备本质上存在模型与数据异质性。现有工作缺乏一种能够在不施加架构限制或假设公共数据可用的前提下适应此类异质性的去中心化联邦学习（DFL）框架。为解决这些问题，我们提出了一种去中心化联邦互学习（DFML）框架，该框架无需中心服务器、支持无限制的异质模型，且避免依赖公共数据。DFML通过客户端间知识蒸馏的互学习机制，以及周期性调整监督信号与蒸馏信号的强度，有效处理模型与数据异质性。大量实验结果表明，DFML在收敛速度和全局准确率方面均表现出稳定的有效性，在各种条件下优于主流基线方法。例如，在CIFAR-100数据集和50个客户端的设置下，DFML在独立同分布（IID）与非独立同分布（non-IID）数据分布场景中，全局准确率分别显著提升了+17.20%和+19.95%。