Federated Learning (FL) enables distributed clients to collaboratively train models without exposing their private data. However, it is difficult to implement efficient FL due to limited resources. Most existing works compress the transmitted gradients or prune the global model to reduce the resource cost, but leave the compressed or pruned parameters under-optimized, which degrades the training performance. To address this issue, the neural composition technique constructs size-adjustable models by composing low-rank tensors, allowing every parameter in the global model to learn the knowledge from all clients. Nevertheless, some tensors can only be optimized by a small fraction of clients, thus the global model may get insufficient training, leading to a long completion time, especially in heterogeneous edge scenarios. To this end, we enhance the neural composition technique, enabling all parameters to be fully trained. Further, we propose a lightweight FL framework, called Heroes, with enhanced neural composition and adaptive local update. A greedy-based algorithm is designed to adaptively assign the proper tensors and local update frequencies for participating clients according to their heterogeneous capabilities and resource budgets. Extensive experiments demonstrate that Heroes can reduce traffic consumption by about 72.05\% and provide up to 2.97$\times$ speedup compared to the baselines.

翻译：联邦学习（FL）使分布式客户端能够在无需暴露私有数据的情况下协作训练模型。然而，由于资源受限，实现高效的FL较为困难。现有大部分工作通过压缩传输梯度或剪枝全局模型来降低资源开销，但导致压缩或剪枝后的参数优化不足，从而降低训练性能。为解决该问题，神经组合技术通过组合低秩张量构建尺寸可调的模型，使全局模型中的每个参数都能从所有客户端学习知识。然而，部分张量只能被少量客户端优化，导致全局模型训练不充分，尤其在异构边缘场景下会延长完成时间。为此，我们改进了神经组合技术，使所有参数得到充分训练。进一步，我们提出名为Heroes的轻量级FL框架，包含增强型神经组合与自适应局部更新。通过设计基于贪心的算法，根据参与客户端异构的能力和资源预算，自适应分配合适的张量与局部更新频率。大量实验表明，与基线方法相比，Heroes可减少约72.05%的流量消耗，并提供高达2.97倍的加速比。