Differential privacy (DP) provides robust privacy guarantees for statistical inference, but this can lead to unreliable results and biases in downstream applications. While several noise-aware approaches have been proposed which integrate DP perturbation into the inference, they are limited to specific types of simple probabilistic models. In this work, we propose a novel method for noise-aware approximate Bayesian inference based on stochastic gradient variational inference which can also be applied to high-dimensional and non-conjugate models. We also propose a more accurate evaluation method for noise-aware posteriors. Empirically, our inference method has similar performance to existing methods in the domain where they are applicable. Outside this domain, we obtain accurate coverages on high-dimensional Bayesian linear regression and well-calibrated predictive probabilities on Bayesian logistic regression with the UCI Adult dataset.

翻译：差分隐私（DP）为统计推断提供了强大的隐私保证，但这可能导致下游应用中产生不可靠的结果和偏差。尽管已有多种噪声感知方法将DP扰动融入推断过程，但它们仅限于特定类型的简单概率模型。本文提出了一种基于随机梯度变分推断的新型噪声感知近似贝叶斯推断方法，该方法同样适用于高维和非共轭模型。我们还提出了一种更精确的噪声感知后验评估方法。实验表明，在现有方法适用的领域，我们的推断方法性能与之相当。而在其适用范围之外，我们针对高维贝叶斯线性回归获得了精确的覆盖率，并针对UCI Adult数据集上的贝叶斯逻辑回归获得了校准良好的预测概率。