The Deep Leakage from Gradient (DLG) attack has emerged as a prevalent and highly effective method for extracting sensitive training data by inspecting exchanged gradients. This approach poses a substantial threat to the privacy of individuals and organizations alike. This research presents a comprehensive analysis of the gradient leakage method when applied specifically to transformer-based models. Through meticulous examination, we showcase the capability to accurately recover data solely from gradients and rigorously investigate the conditions under which gradient attacks can be executed, providing compelling evidence. Furthermore, we reevaluate the approach of introducing additional noise on gradients as a protective measure against gradient attacks. To address this, we outline a theoretical proof that analyzes the associated privacy costs within the framework of differential privacy. Additionally, we affirm the convergence of the Stochastic Gradient Descent (SGD) algorithm under perturbed gradients. The primary objective of this study is to augment the understanding of gradient leakage attack and defense strategies while actively contributing to the development of privacy-preserving techniques specifically tailored for transformer-based models. By shedding light on the vulnerabilities and countermeasures associated with gradient leakage, this research aims to foster advancements in safeguarding sensitive data and upholding privacy in the context of transformer-based models.

翻译：梯度泄露（DLG）攻击作为一种通过审查交换梯度来提取敏感训练数据的普遍且高效方法，对个人及组织隐私构成重大威胁。本研究针对基于Transformer的模型，对梯度泄露方法进行了全面分析。通过细致研究，我们展示了仅从梯度中精确恢复数据的能力，并严谨论证了梯度攻击的实施条件，提供了有力证据。此外，我们重新评估了在梯度上引入额外噪声作为防范梯度攻击措施的有效性。为此，我们提出了一个理论证明，在差分隐私框架下分析了相关隐私成本。同时，我们确认了扰动梯度下随机梯度下降（SGD）算法的收敛性。本研究的主要目标是增进对梯度泄露攻击与防御策略的理解，并积极推动针对Transformer模型的隐私保护技术开发。通过揭示梯度泄露相关的脆弱性与对抗措施，本研究旨在促进敏感数据保护的进步，在Transformer模型背景下维护隐私安全。