In federated learning (FL), clients usually have diverse participation statistics that are unknown a priori, which can significantly harm the performance of FL if not handled properly. Existing works aiming at addressing this problem are usually based on global variance reduction, which requires a substantial amount of additional memory in a multiplicative factor equal to the total number of clients. An important open problem is to find a lightweight method for FL in the presence of clients with unknown participation rates. In this paper, we address this problem by adapting the aggregation weights in federated averaging (FedAvg) based on the participation history of each client. We first show that, with heterogeneous participation statistics, FedAvg with non-optimal aggregation weights can diverge from the optimal solution of the original FL objective, indicating the need of finding optimal aggregation weights. However, it is difficult to compute the optimal weights when the participation statistics are unknown. To address this problem, we present a new algorithm called FedAU, which improves FedAvg by adaptively weighting the client updates based on online estimates of the optimal weights without knowing the statistics of client participation. We provide a theoretical convergence analysis of FedAU using a novel methodology to connect the estimation error and convergence. Our theoretical results reveal important and interesting insights, while showing that FedAU converges to an optimal solution of the original objective and has desirable properties such as linear speedup. Our experimental results also verify the advantage of FedAU over baseline methods with various participation patterns.

翻译：在联邦学习（FL）中，客户端通常具有未知的先验多样参与统计量，若处理不当会显著损害FL性能。现有应对该问题的工作通常基于全局方差缩减，这需要以客户端总数为乘性因子的额外存储空间。一个重要的开放性问题是在客户端参与率未知的情况下寻找FL的轻量级方法。本文通过基于每个客户端的参与历史调整联邦平均（FedAvg）中的聚合权重来解决该问题。我们首先证明，在异构参与统计量下，非最优聚合权重的FedAvg会偏离原始FL目标的最优解，表明需要寻找最优聚合权重。然而，当参与统计量未知时，计算最优权重存在困难。为解决该问题，我们提出新算法FedAU，该算法通过基于最优权重的在线估计自适应地加权客户端更新（无需获知客户端参与统计量）来改进FedAvg。我们采用连接估计误差与收敛性的新型方法论，提供了FedAU的理论收敛性分析。理论结果揭示了重要而有趣的见解，同时表明FedAU收敛到原始目标的最优解，并具有线性加速等理想性质。实验结果也验证了FedAU在不同参与模式下的基线方法相比的优势。