Traditionally, the random noise is equally injected when training with different data instances in the field of differential privacy (DP). In this paper, we first give sharper excess risk bounds of DP stochastic gradient descent (SGD) method. Considering most of the previous methods are under convex conditions, we use Polyak-{\L}ojasiewicz condition to relax it in this paper. Then, after observing that different training data instances affect the machine learning model to different extent, we consider the heterogeneity of training data and attempt to improve the performance of DP-SGD from a new perspective. Specifically, by introducing the influence function (IF), we quantitatively measure the contributions of various training data on the final machine learning model. If the contribution made by a single data instance is so little that attackers cannot infer anything from the model, we do not add noise when training with it. Based on this observation, we design a `Performance Improving' DP-SGD algorithm: PIDP-SGD. Theoretical and experimental results show that our proposed PIDP-SGD improves the performance significantly.

翻译：传统上，在差分隐私（DP）领域的训练过程中，对不同数据实例注入的随机噪声是相同的。本文首先给出了差分隐私随机梯度下降（SGD）方法更精确的过剩风险上界。由于以往多数方法均基于凸性假设，本文采用Polyak-Łojasiewicz条件放宽了这一约束。随后，在观察到不同训练数据实例对机器学习模型影响程度存在差异后，本文从训练数据异质性的新视角出发，尝试提升DP-SGD的性能。具体而言，通过引入影响函数（IF），我们定量衡量了各类训练数据对最终机器学习模型的贡献。若单个数据实例的贡献微乎其微，以至于攻击者无法从模型中推断任何信息，则训练该实例时不添加噪声。基于这一发现，我们设计了一种“性能提升型”DP-SGD算法：PIDP-SGD。理论与实验结果表明，所提出的PIDP-SGD算法显著提升了性能。