Privacy preservation in machine learning, particularly through Differentially Private Stochastic Gradient Descent (DP-SGD), is critical for sensitive data analysis. However, existing statistical inference methods for SGD predominantly focus on cyclic subsampling, while DP-SGD requires randomized subsampling. This paper first bridges this gap by establishing the asymptotic properties of SGD under the randomized rule and extending these results to DP-SGD. For the output of DP-SGD, we show that the asymptotic variance decomposes into statistical, sampling, and privacy-induced components. Two methods are proposed for constructing valid confidence intervals: the plug-in method and the random scaling method. We also perform extensive numerical analysis, which shows that the proposed confidence intervals achieve nominal coverage rates while maintaining privacy.

翻译：机器学习中的隐私保护，特别是通过差分隐私随机梯度下降（DP-SGD），对于敏感数据分析至关重要。然而，现有的SGD统计推断方法主要关注循环子采样，而DP-SGD需要随机子采样。本文首先通过建立随机规则下SGD的渐近性质，并将这些结果扩展到DP-SGD，填补了这一空白。对于DP-SGD的输出，我们证明了渐近方差可分解为统计、采样和隐私诱导三个组成部分。提出了两种构建有效置信区间的方法：插件法和随机尺度法。我们还进行了广泛的数值分析，结果表明所提出的置信区间在保持隐私的同时达到了名义覆盖概率。