Federated learning (FL) is an emerging paradigm to train model with distributed data from numerous Internet of Things (IoT) devices. It inherently assumes a uniform capacity among participants. However, due to different conditions such as differing energy budgets or executing parallel unrelated tasks, participants have diverse computational resources in practice. Participants with insufficient computation budgets must plan for the use of restricted computational resources appropriately, otherwise they would be unable to complete the entire training procedure, resulting in model performance decline. To address the this issue, we propose a strategy for estimating local models without computationally intensive iterations. Based on it, we propose Computationally Customized Federated Averaging (CC-FedAvg), which allows participants to determine whether to perform traditional local training or model estimation in each round based on their current computational budgets. Both theoretical analysis and exhaustive experiments indicate that CC-FedAvg has the same convergence rate and comparable performance as FedAvg without resource constraints. Furthermore, CC-FedAvg can be viewed as a computation-efficient version of FedAvg that retains model performance while considerably lowering computation overhead.
翻译:联邦学习(FL)是一种新兴范式,用于利用来自大量物联网(IoT)设备的分布式数据训练模型。它本质上假设参与者之间具有统一的计算能力。然而,由于能源预算差异或执行并行的无关任务等不同条件,实际中参与者的计算资源存在多样性。计算预算不足的参与者必须合理规划有限计算资源的使用,否则将无法完成整个训练过程,导致模型性能下降。为解决这一问题,我们提出了一种无需密集计算迭代即可估计本地模型的策略。在此基础上,我们提出了计算定制化的联邦平均算法(CC-FedAvg),该算法允许参与者根据当前计算预算,自主决定每轮执行传统本地训练还是模型估计。理论分析与大量实验均表明,CC-FedAvg在无资源约束条件下具有与FedAvg相同的收敛速率和可比的性能。此外,CC-FedAvg可被视为FedAvg的计算高效版本,在保持模型性能的同时显著降低计算开销。