While many solutions for privacy-preserving convex empirical risk minimization (ERM) have been developed, privacy-preserving nonconvex ERM remains a challenge. We study nonconvex ERM, which takes the form of minimizing a finite-sum of nonconvex loss functions over a training set. We propose a new differentially private stochastic gradient descent algorithm for nonconvex ERM that achieves strong privacy guarantees efficiently, and provide a tight analysis of its privacy and utility guarantees, as well as its gradient complexity. Our algorithm reduces gradient complexity while improves the best previous utility guarantee given by Wang et al. (NeurIPS 2017). Our experiments on benchmark nonconvex ERM problems demonstrate superior performance in terms of both training cost and utility gains compared with previous differentially private methods using the same privacy budgets.

翻译：尽管针对隐私保护的凸经验风险最小化（ERM）已发展出诸多解决方案，但隐私保护的非凸ERM仍是一项挑战。我们研究非凸ERM问题，其目标是在训练集上最小化非凸损失函数的有限和。我们提出了一种新的差分隐私随机梯度下降算法，用于非凸ERM，该算法以高效的方式实现了强隐私保证，并对其隐私性、效用性保证以及梯度复杂度进行了严格分析。我们的算法在降低梯度复杂度的同时，提升了Wang等人（NeurIPS 2017）之前提出的最优效用保证。在基准非凸ERM问题上的实验表明，与使用相同隐私预算的先前差分隐私方法相比，我们的方法在训练成本和效用增益方面均表现更优。