In this paper, we propose a first-order distributed optimization algorithm that is provably robust to Byzantine failures-arbitrary and potentially adversarial behavior, where all the participating agents are prone to failure. We model each agent's state over time as a two-state Markov chain that indicates Byzantine or trustworthy behaviors at different time instants. We set no restrictions on the maximum number of Byzantine agents at any given time. We design our method based on three layers of defense: 1) temporal robust aggregation, 2) spatial robust aggregation, and 3) gradient normalization. We study two settings for stochastic optimization, namely Sample Average Approximation and Stochastic Approximation. We provide convergence guarantees of our method for strongly convex and smooth non-convex cost functions.

翻译：在本文中,我们建议了一种第一级分布式优化算法,该算法对拜占庭失败的任意性和潜在对抗行为非常有力,所有参与代理商都容易失败。我们将每个代理商的状态在一段时间内模拟为两州马可夫链条,表明拜占庭或在不同时间的可信赖行为。我们没有对拜占庭代理商在任何特定时间的最大数量设置限制。我们设计我们的方法基于三层防御:(1) 时间稳健总和,(2) 空间稳健总和,(3) 梯度正常化。我们研究了两种随机优化环境,即样本平均匹配和斯托占整。我们保证了我们对于强固和顺畅的非凝固成本功能的方法的趋同性。