Distributed learning (DL) leverages multiple nodes to accelerate training, enabling the efficient optimization of large-scale models. Stochastic Gradient Descent (SGD), a key optimization algorithm, plays a central role in this process. However, communication bottlenecks often limit scalability and efficiency, leading to the increasing adoption of compressed SGD techniques to alleviate these challenges. Despite addressing communication overheads, compressed SGD introduces trustworthiness concerns, as gradient exchanges among nodes are vulnerable to attacks like gradient inversion (GradInv) and membership inference attacks (MIA). The trustworthiness of compressed SGD remains underexplored, leaving important questions about its reliability unanswered. In this paper, we provide a trustworthiness evaluation of compressed versus uncompressed SGD. Specifically, we conduct empirical studies using GradInv attacks, revealing that compressed SGD demonstrates significantly higher resistance to privacy leakage compared to uncompressed SGD. Moreover, our findings suggest that MIA may not be a reliable metric for assessing privacy risks in machine learning.

翻译：分布式学习（DL）通过利用多个节点来加速训练，从而实现大规模模型的高效优化。随机梯度下降（SGD）作为一种核心优化算法，在此过程中发挥着关键作用。然而，通信瓶颈常常限制其可扩展性与效率，这促使压缩SGD技术日益普及以缓解此类挑战。尽管压缩SGD解决了通信开销问题，却引入了可信度方面的隐忧——节点间的梯度交换易受梯度反演（GradInv）和成员推理攻击（MIA）等威胁。目前针对压缩SGD可信度的研究尚不充分，其可靠性仍存在诸多悬而未决的问题。本文对压缩与未压缩SGD进行了可信度评估。具体而言，我们通过GradInv攻击开展实证研究，结果表明：相较于未压缩SGD，压缩SGD对隐私泄露表现出显著更强的抵抗能力。此外，我们的发现暗示MIA可能并非评估机器学习隐私风险的可靠指标。