In this paper, we are concerned with differentially private {stochastic gradient descent (SGD)} algorithms in the setting of stochastic convex optimization (SCO). Most of the existing work requires the loss to be Lipschitz continuous and strongly smooth, and the model parameter to be uniformly bounded. However, these assumptions are restrictive as many popular losses violate these conditions including the hinge loss for SVM, the absolute loss in robust regression, and even the least square loss in an unbounded domain. We significantly relax these restrictive assumptions and establish privacy and generalization (utility) guarantees for private SGD algorithms using output and gradient perturbations associated with non-smooth convex losses. Specifically, the loss function is relaxed to have an $\alpha$-H\"{o}lder continuous gradient (referred to as $\alpha$-H\"{o}lder smoothness) which instantiates the Lipschitz continuity ($\alpha=0$) and the strong smoothness ($\alpha=1$). We prove that noisy SGD with $\alpha$-H\"older smooth losses using gradient perturbation can guarantee $(\epsilon,\delta)$-differential privacy (DP) and attain optimal excess population risk $\mathcal{O}\Big(\frac{\sqrt{d\log(1/\delta)}}{n\epsilon}+\frac{1}{\sqrt{n}}\Big)$, up to logarithmic terms, with the gradient complexity $ \mathcal{O}( n^{2-\alpha\over 1+\alpha}+ n).$ This shows an important trade-off between $\alpha$-H\"older smoothness of the loss and the computational complexity for private SGD with statistically optimal performance. In particular, our results indicate that $\alpha$-H\"older smoothness with $\alpha\ge {1/2}$ is sufficient to guarantee $(\epsilon,\delta)$-DP of noisy SGD algorithms while achieving optimal excess risk with the linear gradient complexity $\mathcal{O}(n).$

翻译：在本文中, 我们关注在建立 comvex 优化( SCO) 中, 私人的 {stochastectial lider_ lider_ lider_ lider_ compex 优化( SCO) 中, 我们关注的算法有差异性 {stochatical lider_ lider_ lider{ lider_ lider_ lider_ lider_ liform} 。 具体地说, 损失功能是放松的, 要有一个 $\ alphar_ H\\\ lider_ lider_ lider_ liver 这样的条件, 包括SVMmission 的断层损失, 强性能使 Lipschite 的连续性(\ dalphal=0) 和强性平滑度( AL=1美元) 。 我们证明, 以 $\\\\\\\\\\\\\\\ lical deal dealalal dealalalal lial list list modeal sal=xal modeal sal lax sal modeal_ mox modeal_ lixal_ lax sal_ =_ =============================s salsalsals=================sil sal smodeal sal salsil sal sal sal sal sal sal sal sal sal sal sal sal sal sal mal mal sal mal sal sal sal sal sal sal sal sal sal sal sal sal sal sal sal sal sal sal sal sal movall