We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latency--comprising both computation and communication latency. Analytical results on both the convergence speed and the DP level are derived for both a skipping scheme (which ignores the stragglers after a timeout) and a baseline scheme that waits for each node to finish before the training continues. A trade-off between overall training latency, accuracy, and privacy, parameterized by the timeout of the skipping scheme, is identified and empirically validated for logistic regression on a real-world dataset and for image classification using the MNIST and CIFAR-10 datasets.

翻译：我们研究了逻辑环结构下去中心化学习中的滞后节点问题，同时确保用户数据隐私。具体而言，我们将Cyffers与Bellet近期提出的差分隐私（DP）去中心化放大框架进行扩展，以纳入包含计算延迟与通信延迟在内的整体训练时延。针对跳过策略（超时后忽略滞后节点）和基线策略（等待所有节点完成后继续训练），我们分别推导了收敛速度与差分隐私保护水平的解析结果。发现跳过策略的超时参数会带来整体训练时延、准确率与隐私之间的权衡，并通过真实数据集上的逻辑回归任务以及MNIST与CIFAR-10数据集上的图像分类任务进行了实证验证。