Differential privacy has become a cornerstone in the development of privacy-preserving learning algorithms. This work addresses optimizing differentially private kernel learning within the empirical risk minimization (ERM) framework. We propose a novel differentially private kernel ERM algorithm based on random projection in the reproducing kernel Hilbert space using Gaussian processes. Our method achieves minimax-optimal excess risk rates for both the squared loss and Lipschitz-smooth convex loss functions under a local strong convexity condition. We further show that existing approaches based on alternative dimension reduction techniques, such as random Fourier feature mappings or $\ell_2$ regularization, yield suboptimal excess risk bounds. Our key theoretical contribution also includes the derivation of dimension-free excess risk bounds for objective perturbation-based private linear ERM, marking the first such result that does not rely on noisy gradient-based mechanisms. Additionally, we obtain sharper excess risk bounds for existing differentially private kernel ERM algorithms. Empirical evaluations support our theoretical claims, demonstrating that random projection enables statistically efficient and optimally private kernel learning. These findings provide new insights into the design of differentially private algorithms and highlight the central role of dimension reduction in balancing privacy and utility.

翻译：差分隐私已成为隐私保护学习算法发展的基石。本研究针对经验风险最小化框架下的差分隐私核学习优化问题展开。我们提出了一种基于再生核希尔伯特空间中高斯过程随机投影的新型差分隐私核经验风险最小化算法。在局部强凸条件下，对于平方损失函数和Lipschitz光滑凸损失函数，该方法实现了极小化最优过剩风险率。我们进一步证明，基于随机傅里叶特征映射或ℓ2正则化等其他降维技术的现有方法，会产生次优的过剩风险界。我们的关键理论贡献还包括为目标扰动型私有线性经验风险最小化推导了维数无关的过剩风险界，这是首个不依赖噪声梯度机制的相关结果。此外，我们为现有的差分隐私核经验风险最小化算法获得了更紧的过剩风险界。实证评估支持了我们的理论结论，表明随机投影能实现统计高效且最优私密的核学习。这些发现为差分隐私算法的设计提供了新见解，并凸显了降维在平衡隐私与效用中的核心作用。