In federated learning (FL), clients usually have diverse participation probabilities that are unknown a priori, which can significantly harm the performance of FL if not handled properly. Existing works aiming at addressing this problem are usually based on global variance reduction, which requires a substantial amount of additional memory in a multiplicative factor equal to the total number of clients. An important open problem is to find a lightweight method for FL in the presence of clients with unknown participation rates. In this paper, we address this problem by adapting the aggregation weights in federated averaging (FedAvg) based on the participation history of each client. We first show that, with heterogeneous participation probabilities, FedAvg with non-optimal aggregation weights can diverge from the optimal solution of the original FL objective, indicating the need of finding optimal aggregation weights. However, it is difficult to compute the optimal weights when the participation probabilities are unknown. To address this problem, we present a new algorithm called FedAU, which improves FedAvg by adaptively weighting the client updates based on online estimates of the optimal weights without knowing the probabilities of client participation. We provide a theoretical convergence analysis of FedAU using a novel methodology to connect the estimation error and convergence. Our theoretical results reveal important and interesting insights, while showing that FedAU converges to an optimal solution of the original objective and has desirable properties such as linear speedup. Our experimental results also verify the advantage of FedAU over baseline methods.

翻译：在联邦学习（FL）中，客户端通常具有未知的先验参与概率，若处理不当，将严重损害FL的性能。现有解决该问题的方法通常基于全局方差缩减，这需要额外存储量，且存储量与客户端总数呈乘法关系增长。一个重要的开放问题是如何在客户端参与率未知的情况下，为FL设计轻量级方法。本文通过基于每个客户端的参与历史调整联邦平均（FedAvg）中的聚合权重来解决该问题。我们首先证明，在参与概率异构的情况下，使用非最优聚合权重的FedAvg可能偏离原始FL目标的最优解，这表明寻找最优聚合权重的必要性。然而，当参与概率未知时，计算最优权重十分困难。为此，我们提出一种新算法FedAU，该算法无需知晓客户端参与概率，通过在线估计最优权重自适应地加权客户端更新，从而改进FedAvg。我们采用一种连接估计误差与收敛性的新方法论，给出了FedAU的理论收敛性分析。理论结果揭示了重要且有趣的见解，同时表明FedAU收敛到原始目标函数的最优解，并具有线性加速等理想性质。实验结果也验证了FedAU相较于基线方法的优越性。