In the realm of real-world devices, centralized servers in Federated Learning (FL) present challenges including communication bottlenecks and susceptibility to a single point of failure. Additionally, contemporary devices inherently exhibit model and data heterogeneity. Existing work lacks a Decentralized FL (DFL) framework capable of accommodating such heterogeneity without imposing architectural restrictions or assuming the availability of public data. To address these issues, we propose a Decentralized Federated Mutual Learning (DFML) framework that is serverless, supports nonrestrictive heterogeneous models, and avoids reliance on public data. DFML effectively handles model and data heterogeneity through mutual learning, which distills knowledge between clients, and cyclically varying the amount of supervision and distillation signals. Extensive experimental results demonstrate consistent effectiveness of DFML in both convergence speed and global accuracy, outperforming prevalent baselines under various conditions. For example, with the CIFAR-100 dataset and 50 clients, DFML achieves a substantial increase of +17.20% and +19.95% in global accuracy under Independent and Identically Distributed (IID) and non-IID data shifts, respectively.

翻译：在真实设备环境中，联邦学习（FL）中的中心化服务器存在通信瓶颈和单点故障风险等挑战。此外，当代设备天然具有模型异构性与数据异构性。现有研究缺乏一种无需限制架构或假设存在公开数据集、即可适配此类异构性的去中心化联邦学习（DFL）框架。为解决上述问题，我们提出去中心化联邦互学习（DFML）框架，该框架无服务器架构、支持无限制异构模型、且不依赖公开数据集。DFML通过互学习机制（在客户端间进行知识蒸馏）以及循环交替调整监督信号与蒸馏信号的比例，有效处理模型与数据异构性。广泛实验结果表明，DFML在收敛速度与全局精度上均具有持续有效性，在多种条件下优于主流基准方法。例如，在CIFAR-100数据集与50个客户端场景下，DFML在独立同分布（IID）与非独立同分布（non-IID）数据偏移下分别实现了全局精度+17.20%与+19.95%的显著提升。