We address the challenge of sample efficiency in differentially private fine-tuning of large language models (LLMs) using DP-SGD. While DP-SGD provides strong privacy guarantees, the added noise significantly increases the entropy of gradient matrices, disrupting their low-rank structure and slowing optimization. We propose a post-processing algorithm that leverages random matrix theory to denoise gradients, restore low-rank structure, and improve alignment with the original signal. Applied to DP-SGD fine-tuning of RoBERTa on GLUE tasks, our method improves sample efficiency compared to state-of-the-art approaches, substantially reducing training time when optimal performance is not required. This work demonstrates that matrix recovery techniques can enhance the utility of private language model training without compromising privacy guarantees.

翻译：我们解决了使用差分隐私随机梯度下降（DP-SGD）对大型语言模型（LLM）进行差分隐私微调时的样本效率挑战。虽然DP-SGD提供了强大的隐私保证，但所添加的噪声显著增加了梯度矩阵的熵，破坏了其低秩结构并减缓了优化过程。我们提出了一种后处理算法，该算法利用随机矩阵理论对梯度进行去噪，恢复其低秩结构，并改善其与原始信号的对齐。将我们的方法应用于RoBERTa在GLUE任务上的DP-SGD微调时，相较于现有最优方法，它提高了样本效率，并在不需要达到最优性能的情况下显著减少了训练时间。这项工作表明，矩阵恢复技术可以在不损害隐私保证的前提下，提升隐私保护语言模型训练的实用性。