Empirical risk minimization (ERM) is a cornerstone of modern machine learning (ML), supported by advances in optimization theory that ensure efficient solutions with provable algorithmic convergence rates, which measure the speed at which optimization algorithms approach a solution, and statistical learning rates, which characterize how well the solution generalizes to unseen data. Privacy, memory, computational, and communications constraints increasingly necessitate data collection, processing, and storage across network-connected devices. In many applications, these networks operate in decentralized settings where a central server cannot be assumed, requiring decentralized ML algorithms that are both efficient and resilient. Decentralized learning, however, faces significant challenges, including an increased attack surface for adversarial interference during decentralized learning processes. This paper focuses on the man-in-the-middle (MITM) attack, which can cause models to deviate significantly from their intended ERM solutions. To address this challenge, we propose RESIST (Resilient dEcentralized learning using conSensus gradIent deScenT), an optimization algorithm designed to be robust against adversarially compromised communication links. RESIST achieves algorithmic and statistical convergence for strongly convex, Polyak-Lojasiewicz, and nonconvex ERM problems. Experimental results demonstrate the robustness and scalability of RESIST for real-world decentralized learning in adversarial environments.

翻译：经验风险最小化（ERM）是现代机器学习的基石，其背后有优化理论的支持，该理论保证了算法能以可证明的收敛速度高效求解——收敛速度衡量优化算法逼近解的快慢，而统计学习速率则刻画了该解对未见数据的泛化能力。隐私、内存、计算和通信限制日益要求数据在网络连接的设备间进行采集、处理和存储。在许多应用中，这些网络运行于去中心化环境中，无法假设存在中央服务器，因此需要既高效又具备弹性的去中心化机器学习算法。然而，去中心化学习面临着严峻挑战，包括在去中心化学习过程中对抗性干扰的攻击面扩大。本文重点关注中间人攻击，该攻击可导致模型显著偏离其预期的ERM解。为应对这一挑战，我们提出RESIST（基于共识梯度下降的弹性去中心化学习），这是一种针对对抗性受损通信链路具有鲁棒性的优化算法。RESIST在强凸、Polyak-Lojasiewicz及非凸的ERM问题上均实现了算法与统计收敛。实验结果表明，RESIST在对抗性环境中进行实际去中心化学习时具备鲁棒性与可扩展性。