As distributed learning applications such as Federated Learning, the Internet of Things (IoT), and Edge Computing grow, it is critical to address the shortcomings of such technologies from a theoretical perspective. As an abstraction, we consider decentralized learning over a network of communicating clients or nodes and tackle two major challenges: data heterogeneity and adversarial robustness. We propose a decentralized minimax optimization method that employs two important modules: local updates and gradient tracking. Minimax optimization is the key tool to enable adversarial training for ensuring robustness. Having local updates is essential in Federated Learning (FL) applications to mitigate the communication bottleneck, and utilizing gradient tracking is essential to proving convergence in the case of data heterogeneity. We analyze the performance of the proposed algorithm, Dec-FedTrack, in the case of nonconvex-strongly concave minimax optimization, and prove that it converges a stationary point. We also conduct numerical experiments to support our theoretical findings.

翻译：随着联邦学习、物联网（IoT）和边缘计算等分布式学习应用的不断发展，从理论视角解决这些技术的固有缺陷变得至关重要。作为抽象模型，我们考虑在由通信客户端或节点组成的网络中进行去中心化学习，并应对两大核心挑战：数据异构性和对抗鲁棒性。我们提出一种结合两个关键模块（局部更新与梯度追踪）的去中心化极小极大优化方法。极小极大优化是实现对抗训练以确保鲁棒性的核心工具；联邦学习（FL）应用中必须引入局部更新以缓解通信瓶颈，而利用梯度追踪对于在数据异构情况下证明收敛性至关重要。我们分析了所提算法Dec-FedTrack在非凸-强凹极小极大优化场景中的性能，并证明其收敛至稳定点。此外，我们通过数值实验验证了理论结论。