Distributed optimization with open collaboration is a popular field since it provides an opportunity for small groups/companies/universities, and individuals to jointly solve huge-scale problems. However, standard optimization algorithms are fragile in such settings due to the possible presence of so-called Byzantine workers -- participants that can send (intentionally or not) incorrect information instead of the one prescribed by the protocol (e.g., send anti-gradient instead of stochastic gradients). Thus, the problem of designing distributed methods with provable robustness to Byzantine workers has been receiving a lot of attention recently. In particular, several works consider a very promising way to achieve Byzantine tolerance via exploiting variance reduction and robust aggregation. The existing approaches use SAGA- and SARAH-type variance-reduced estimators, while another popular estimator -- SVRG -- is not studied in the context of Byzantine-robustness. In this work, we close this gap in the literature and propose a new method -- Byzantine-Robust Loopless Stochastic Variance Reduced Gradient (BR-LSVRG). We derive non-asymptotic convergence guarantees for the new method in the strongly convex case and compare its performance with existing approaches in numerical experiments.

翻译：开放协作的分布式优化是一个热门领域，因为它为小型团体、公司、大学以及个人提供了共同解决超大规模问题的机会。然而，标准优化算法在此类设定下是脆弱的，因为可能存在所谓的拜占庭工作节点——这些参与者会（有意或无意地）发送错误信息，而非协议规定的信息（例如，发送反梯度而非随机梯度）。因此，设计具有可证明拜占庭鲁棒性的分布式方法的问题近来受到广泛关注。特别是，多项工作考虑了通过利用方差缩减和鲁棒聚合来实现拜占庭容忍这一非常有前景的路径。现有方法采用了SAGA和SARAH类型的方差缩减估计器，而另一种流行的估计器——SVRG——尚未在拜占庭鲁棒性语境下被研究。在这项工作中，我们填补了这一文献空白，并提出了一种新方法——拜占庭鲁棒无环随机方差缩减梯度（BR-LSVRG）。我们在强凸情形下推导了新方法的非渐近收敛保证，并在数值实验中将其性能与现有方法进行了比较。