Differentially private stochastic gradient descent (DP-SGD) adds noise to gradients in back-propagation, safeguarding training data from privacy leakage, particularly membership inference. It fails to cover (inference-time) threats like embedding inversion and sensitive attribute inference. It is also costly in storage and computation when used to fine-tune large pre-trained language models (LMs). We propose DP-Forward, which directly perturbs embedding matrices in the forward pass of LMs. It satisfies stringent local DP requirements for training and inference data. To instantiate it using the smallest matrix-valued noise, we devise an analytic matrix Gaussian~mechanism (aMGM) by drawing possibly non-i.i.d. noise from a matrix Gaussian distribution. We then investigate perturbing outputs from different hidden (sub-)layers of LMs with aMGM noises. Its utility on three typical tasks almost hits the non-private baseline and outperforms DP-SGD by up to 7.7pp at a moderate privacy level. It saves 3$\times$ time and memory costs compared to DP-SGD with the latest high-speed library. It also reduces the average success rates of embedding inversion and sensitive attribute inference by up to 88pp and 41pp, respectively, whereas DP-SGD fails.

翻译：差分隐私随机梯度下降（DP-SGD）在反向传播中向梯度添加噪声，以保护训练数据免受隐私泄露（尤其是成员推理攻击），但无法覆盖嵌入反演和敏感属性推断等（推理时）威胁。当用于微调大型预训练语言模型（LM）时，其在存储和计算方面成本高昂。我们提出DP-Forward方法，该方法直接扰动LM前向传播中的嵌入矩阵，以满足训练和推理数据的严格局部差分隐私要求。为使用最小矩阵值噪声实现该方法，我们设计了一种解析矩阵高斯机制（aMGM），通过从矩阵高斯分布中抽取可能非独立同分布的噪声来实现。随后，我们研究了用aMGM噪声扰动LM不同隐藏（子）层输出。在三个典型任务上，其效用几乎达到非隐私基线水平，并在中等隐私水平下，性能比DP-SGD高出最多7.7个百分点。与使用最新高速库的DP-SGD相比，它节省了3倍的时间和内存成本。此外，DP-Forward将嵌入反演和敏感属性推断的平均成功率分别降低了最多88个百分点和41个百分点，而DP-SGD在此方面无效。