Differentially private stochastic gradient descent (DP-SGD) is the most widely used method for training machine learning models with provable privacy guarantees. A key challenge in DP-SGD is setting the per-sample gradient clipping threshold, which significantly affects the trade-off between privacy and utility. While recent adaptive methods improve performance by adjusting this threshold during training, they operate in the standard coordinate system and fail to account for correlations across the coordinates of the gradient. We propose GeoClip, a geometry-aware framework that clips and perturbs gradients in a transformed basis aligned with the geometry of the gradient distribution. GeoClip adaptively estimates this transformation using only previously released noisy gradients, incurring no additional privacy cost. We provide convergence guarantees for GeoClip and derive a closed-form solution for the optimal transformation that minimizes the amount of noise added while keeping the probability of gradient clipping under control. Experiments on both tabular and image datasets demonstrate that GeoClip consistently outperforms existing adaptive clipping methods under the same privacy budget.

翻译：差分隐私随机梯度下降（DP-SGD）是当前训练具备可证明隐私保障的机器学习模型时最广泛采用的方法。DP-SGD中的一个关键挑战在于设定逐样本梯度裁剪阈值，该阈值显著影响隐私性与效用之间的权衡。尽管近期提出的自适应方法通过在训练过程中动态调整该阈值以提升性能，但这些方法均在标准坐标系下操作，未能考虑梯度各维度间的相关性。本文提出GeoClip，一种几何感知框架，该框架在依据梯度分布几何结构对齐的变换基中对梯度进行裁剪与扰动。GeoClip仅利用先前已发布的带噪梯度自适应估计该变换，无需额外隐私开销。我们为GeoClip提供了收敛性保证，并推导出最优变换的闭式解，该解能在控制梯度裁剪概率的同时最小化所添加噪声量。在表格数据与图像数据集上的实验表明，在相同隐私预算下，GeoClip始终优于现有的自适应裁剪方法。