Most prior convergence results on differentially private stochastic gradient descent (DP-SGD) are derived under the simplistic assumption of uniform Lipschitzness, i.e., the per-sample gradients are uniformly bounded. This assumption is unrealistic in many problems, e.g., linear regression with Gaussian data. We relax uniform Lipschitzness by instead assuming that the per-sample gradients have \textit{sample-dependent} upper bounds, i.e., per-sample Lipschitz constants, which themselves may be unbounded. We derive new convergence results for DP-SGD on both convex and nonconvex functions when the per-sample Lipschitz constants have bounded moments. Furthermore, we provide principled guidance on choosing the clip norm in DP-SGD for convex settings satisfying our relaxed version of Lipschitzness, without making distributional assumptions on the Lipschitz constants. We verify the effectiveness of our recommendation via experiments on benchmarking datasets.

翻译：在不同的私人悬浮梯度下降(DP-SGD)方面,大多数先前的趋同结果来自单一的Lipschitzness(PDP-SGD)简单假设,即每样梯度是统一的,每个样梯度是统一的,这种假设在许多问题上是不切实际的,例如,用Gaussian数据进行线性回归。我们放松了统一的Lipschitzness(DP-SGD),办法是假设每个样梯度的上界,即每个样梯度的每个样状Lipschitz常数,这些常数本身可能不受约束。当每样样梯度常数相交时,我们为DP-SGD的 convex和非convex函数得出新的趋同结果。此外,我们提供原则性指导,说明如何选择DP-SGD的剪辑规范来满足我们放松版的Lipschitznations,而不在Lipschitz常数上作出分布性假设。我们通过基准数据集试验来核查我们的建议的有效性。