Federated learning (FL) holds strong potential for distributed machine learning, but in heterogeneous environments, Bandwidth-Constrained Clients (BCCs) often struggle to participate effectively due to limited communication capacity. Their small sub-models learn quickly at first but become under-parameterized in later stages, leading to slow convergence and degraded generalization. We propose FedGMR - Federated Learning with Gradual Model Restoration under Asynchrony and Model Heterogeneity. FedGMR progressively increases each client's sub-model density during training, enabling BCCs to remain effective contributors throughout the process. In addition, we develop a mask-aware aggregation rule tailored for asynchronous MHFL and provide convergence guarantees showing that aggregated error scales with the average sub-model density across clients and rounds, while GMR provably shrinks this gap toward full-model FL. Extensive experiments on FEMNIST, CIFAR-10, and ImageNet-100 demonstrate that FedGMR achieves faster convergence and higher accuracy, especially under high heterogeneity and non-IID settings.

翻译：联邦学习（FL）在分布式机器学习中具有巨大潜力，但在异构环境下，带宽受限客户端（BCCs）常因通信能力有限而难以有效参与。其小型子模型在初始阶段学习迅速，但在后期阶段因参数不足而表现不佳，导致收敛缓慢和泛化能力下降。本文提出FedGMR——异步与模型异构下的渐进式模型恢复联邦学习。FedGMR在训练过程中逐步增加各客户端的子模型密度，使BCCs能够全程保持有效贡献。此外，我们开发了一种专为异步异构联邦学习设计的掩码感知聚合规则，并提供收敛性证明，表明聚合误差与客户端和轮次的平均子模型密度呈比例关系，而GMR可证明将该误差差距缩小至接近全模型联邦学习水平。在FEMNIST、CIFAR-10和ImageNet-100数据集上的大量实验表明，FedGMR实现了更快的收敛速度和更高的准确率，尤其在高度异构和非独立同分布场景下表现突出。