In this paper, we study the challenging task of Byzantine-robust decentralized training on arbitrary communication graphs. Unlike federated learning where workers communicate through a server, workers in the decentralized environment can only talk to their neighbors, making it harder to reach consensus and benefit from collaborative training. To address these issues, we propose a ClippedGossip algorithm for Byzantine-robust consensus and optimization, which is the first to provably converge to a $O(\delta_{\max}\zeta^2/\gamma^2)$ neighborhood of the stationary point for non-convex objectives under standard assumptions. Finally, we demonstrate the encouraging empirical performance of ClippedGossip under a large number of attacks.

翻译：本文研究了在任意通信图上进行拜占庭鲁棒去中心化训练这一具有挑战性的任务。与通过服务器通信的联邦学习不同，去中心化环境中的工作节点只能与相邻节点通信，这使得达成共识和从协作训练中获益变得更加困难。为解决这些问题，我们提出了一种用于拜占庭鲁棒共识与优化的ClippedGossip算法，该算法是首个在标准假设下，针对非凸目标函数能够证明收敛到驻点$O(\delta_{\max}\zeta^2/\gamma^2)$邻域的方案。最后，我们展示了ClippedGossip算法在大量攻击下令人鼓舞的实证性能。