Recently, due to the popularity of deep neural networks and other methods whose training typically relies on the optimization of an objective function, and due to concerns for data privacy, there is a lot of interest in differentially private gradient descent methods. To achieve differential privacy guarantees with a minimum amount of noise, it is important to be able to bound precisely the sensitivity of the information which the participants will observe. In this study, we present a novel approach that mitigates the bias arising from traditional gradient clipping. By leveraging public information concerning the current global model and its location within the search domain, we can achieve improved gradient bounds, leading to enhanced sensitivity determinations and refined noise level adjustments. We extend the state of the art algorithms, present improved differential privacy guarantees requiring less noise and present an empirical evaluation.

翻译：最近，由于深度神经网络等依赖目标函数优化进行训练的模型日益普及，加之数据隐私问题的关注度提升，差分隐私梯度下降方法引起了广泛关注。为在最小噪声条件下实现差分隐私保障，精确界定参与者可观测信息的敏感度至关重要。本研究提出了一种新方法，可有效缓解传统梯度裁剪引起的偏差。通过利用关于当前全局模型及其在搜索域中位置公共信息，我们能够获得更优的梯度边界，从而提升敏感度判定精度并优化噪声水平调节。我们拓展了现有最优算法，提出了需要更少噪声的改进型差分隐私保障机制，并通过实证评估验证了其有效性。