This paper focuses on decentralized stochastic optimization in the presence of Byzantine attacks. During the optimization process, an unknown number of malfunctioning or malicious workers, termed as Byzantine workers, disobey the algorithmic protocol and send arbitrarily wrong messages to their neighbors. Even though various Byzantine-resilient algorithms have been developed for distributed stochastic optimization with a central server, we show that there are two major issues in the existing robust aggregation rules when being applied to the decentralized scenario: disagreement and non-doubly stochastic virtual mixing matrix. This paper provides comprehensive analysis that discloses the negative effects of these two issues, and gives guidelines of designing favorable Byzantine-resilient decentralized stochastic optimization algorithms. Under these guidelines, we propose iterative outlier scissor (IOS), an iterative filtering-based robust aggregation rule with provable performance guarantees. Numerical experiments demonstrate the effectiveness of IOS. The code of simulation implementation is available at github.com/Zhaoxian-Wu/IOS.

翻译：本文聚焦于存在拜占庭攻击时的去中心化随机优化问题。在优化过程中，未知数量的故障或恶意工作节点（称为拜占庭节点）违背算法协议，向邻居节点发送任意错误信息。尽管针对带有中心服务器的分布式随机优化已开发出多种拜占庭弹性算法，但本文指出现有鲁棒聚合规则在应用于去中心化场景时存在两大核心问题：不一致性及非双随机的虚拟混合矩阵。本文通过全面分析揭示了这两类问题的负面影响，并给出了设计高性能拜占庭弹性去中心化随机优化算法的指导原则。基于这些原则，我们提出迭代离群点剪裁器（IOS）——一种基于迭代滤波的鲁棒聚合规则，具备可证明的性能保证。数值实验验证了IOS的有效性。仿真实现的代码可在 github.com/Zhaoxian-Wu/IOS 获取。