Federated learning (FL) is a decentralized learning framework wherein a parameter server (PS) and a collection of clients collaboratively train a model via minimizing a global objective. Communication bandwidth is a scarce resource; in each round, the PS aggregates the updates from a subset of clients only. In this paper, we focus on non-convex minimization that is vulnerable to non-uniform and time-varying communication failures between the PS and the clients. Specifically, in each round $t$, the link between the PS and client $i$ is active with probability $p_i^t$, which is $\textit{unknown}$ to both the PS and the clients. This arises when the channel conditions are heterogeneous across clients and are changing over time. We show that when the $p_i^t$'s are not uniform, $\textit{Federated Average}$ (FedAvg) -- the most widely adopted FL algorithm -- fails to minimize the global objective. Observing this, we propose $\textit{Federated Postponed Broadcast}$ (FedPBC) which is a simple variant of FedAvg. It differs from FedAvg in that the PS postpones broadcasting the global model till the end of each round. We show that FedPBC converges to a stationary point of the original objective. The introduced staleness is mild and there is no noticeable slowdown. Both theoretical analysis and numerical results are provided. On the technical front, postponing the global model broadcasts enables implicit gossiping among the clients with active links at round $t$. Despite $p_i^t$'s are time-varying, we are able to bound the perturbation of the global model dynamics via the techniques of controlling the gossip-type information mixing errors.

翻译：联邦学习（FL）是一种去中心化学习框架，其中参数服务器（PS）与一组客户端通过最小化全局目标协同训练模型。通信带宽是稀缺资源；在每一轮中，PS仅聚合来自部分客户端的更新。本文重点关注非凸最小化问题，该问题易受PS与客户端之间非均匀且时变的通信故障影响。具体而言，在每一轮$t$中，PS与客户端$i$之间的链路以概率$p_i^t$处于活跃状态，而该概率$p_i^t$对PS和客户端均为$\textit{未知}$。当信道条件在客户端间呈现异质性并随时间变化时，便会出现此类情况。我们证明，当$p_i^t$非均匀时，最广泛采用的FL算法——$\textit{联邦平均}$（FedAvg）无法最小化全局目标。鉴于此，我们提出$\textit{联邦延迟广播}$（FedPBC），这是FedAvg的一种简单变体。与FedAvg的不同之处在于，PS将全局模型的广播推迟至每一轮结束时。我们证明，FedPBC能够收敛至原始目标的驻点。引入的滞后性较为轻微，且未观察到明显减速。本文提供了理论分析与数值实验结果。在技术层面，延迟全局模型广播使得在第$t$轮中拥有活跃链路的客户端之间能够实现隐式信息交换。尽管$p_i^t$是时变的，我们仍能通过控制信息交换类混合误差的技术手段，对全局模型动态的扰动进行界定量化。