Differential privacy has become a cornerstone in the development of privacy-preserving learning algorithms. This work addresses optimizing differentially private kernel learning within the empirical risk minimization (ERM) framework. We propose a novel differentially private kernel ERM algorithm based on random projection in the reproducing kernel Hilbert space using Gaussian processes. Our method achieves minimax-optimal excess risk for both the squared loss and Lipschitz-smooth convex loss functions under a local strong convexity condition. We further show that existing approaches based on alternative dimension reduction techniques, such as random Fourier feature mappings or $\ell_2$ regularization, yield suboptimal generalization performance. Our key theoretical contribution also includes the derivation of dimension-free generalization bounds for objective perturbation-based private linear ERM -- marking the first such result that does not rely on noisy gradient-based mechanisms. Additionally, we obtain sharper generalization bounds for existing differentially private kernel ERM algorithms. Empirical evaluations support our theoretical claims, demonstrating that random projection enables statistically efficient and optimally private kernel learning. These findings provide new insights into the design of differentially private algorithms and highlight the central role of dimension reduction in balancing privacy and utility.

翻译：差分隐私已成为隐私保护学习算法发展的基石。本研究致力于在经验风险最小化框架下优化差分隐私核学习。我们提出了一种基于再生核希尔伯特空间中高斯过程随机投影的新型差分隐私核经验风险最小化算法。在局部强凸性条件下，我们的方法对于平方损失和Lipschitz光滑凸损失函数均达到了极小极大最优超额风险。我们进一步证明，基于其他降维技术（如随机傅里叶特征映射或$\ell_2$正则化）的现有方法会产生次优泛化性能。我们的核心理论贡献还包括推导了基于目标扰动的隐私线性经验风险最小化的无维度泛化界——这是首个不依赖噪声梯度机制的此类结果。此外，我们为现有差分隐私核经验风险最小化算法获得了更尖锐的泛化界。实证评估支持我们的理论主张，表明随机投影能够实现统计高效且最优隐私的核学习。这些发现为差分隐私算法的设计提供了新见解，并凸显了降维技术在平衡隐私与效用中的核心作用。