Federated averaging (FedAvg) is a communication efficient algorithm for the distributed training with an enormous number of clients. In FedAvg, clients keep their data locally for privacy protection; a central parameter server is used to communicate between clients. This central server distributes the parameters to each client and collects the updated parameters from clients. FedAvg is mostly studied in centralized fashions, which requires massive communication between server and clients in each communication. Moreover, attacking the central server can break the whole system's privacy. In this paper, we study the decentralized FedAvg with momentum (DFedAvgM), which is implemented on clients that are connected by an undirected graph. In DFedAvgM, all clients perform stochastic gradient descent with momentum and communicate with their neighbors only. To further reduce the communication cost, we also consider the quantized DFedAvgM. We prove convergence of the (quantized) DFedAvgM under trivial assumptions; the convergence rate can be improved when the loss function satisfies the P{\L} property. Finally, we numerically verify the efficacy of DFedAvgM.

翻译：平均( FedAvg) 是一种与大量客户进行分布式培训的通信高效算法。 在 FedAvg 中, 客户将数据保存在本地用于隐私保护; 中央参数服务器用于客户之间的通信。 这个中央服务器向每个客户分发参数, 并从客户收集更新参数。 FedAvg 大多以集中方式研究, 这要求服务器和客户在每次通信中进行大规模通信。 此外, 攻击中央服务器可以打破整个系统的隐私。 在本文中, 我们研究分散式的FedAvgg 与动力( DFedAvgM ), 该动力是用无方向图形连接的客户执行的。 在 DFedAvgM 中, 所有客户都以动态进行随机梯度梯度梯度下降, 并且只与邻居进行通信。 为了进一步降低通信成本, 我们还考虑在微小的假设下对DFedAvgM 进行量化的合并。 当损失功能满足 PL} 属性时, 组合率是可以改进的。 最后, 我们用数字核查DFDFAgM 的功效。