Modern machine learning (ML) models are capable of impressive performances. However, their prowess is not due only to the improvements in their architecture and training algorithms but also to a drastic increase in computational power used to train them. Such a drastic increase led to a growing interest in distributed ML, which in turn made worker failures and adversarial attacks an increasingly pressing concern. While distributed byzantine resilient algorithms have been proposed in a differentiable setting, none exist in a gradient-free setting. The goal of this work is to address this shortcoming. For that, we introduce a more general definition of byzantine-resilience in ML - the \textit{model-consensus}, that extends the definition of the classical distributed consensus. We then leverage this definition to show that a general class of gradient-free ML algorithms - ($1,\lambda$)-Evolutionary Search - can be combined with classical distributed consensus algorithms to generate gradient-free byzantine-resilient distributed learning algorithms. We provide proofs and pseudo-code for two specific cases - the Total Order Broadcast and proof-of-work leader election.

翻译：现代机器学习模型展示了令人瞩目的性能。然而，其强大能力不仅源于架构与训练算法的改进，更得益于训练所用计算能力的急剧提升。这种计算能力的激增引发了对分布式机器学习日益浓厚的兴趣，进而使得工作节点故障和对抗性攻击成为愈发紧迫的挑战。尽管在可微场景下已有分布式拜占庭鲁棒算法的相关研究，但在无梯度场景中尚无此类算法。本研究旨在填补这一空白。为此，我们提出机器学习中更一般的拜占庭鲁棒定义——模型共识，该定义扩展了经典分布式共识的概念。基于这一定义，我们论证了通用无梯度机器学习算法族——(1,λ)-进化搜索——可与经典分布式共识算法结合，从而生成无梯度的拜占庭鲁棒分布式学习算法。我们针对全序广播和工作量证明领导者选举这两种具体场景，提供了理论证明与伪代码实现。