Bayesian methods lie at the heart of modern data science and provide a powerful scaffolding for estimation in data-constrained settings and principled quantification and propagation of uncertainty. Yet in many real-world use cases where these methods are deployed, there is a natural need to preserve the privacy of the individuals whose data is being scrutinized. While a number of works have attempted to approach the problem of differentially private Bayesian estimation through either reasoning about the inherent privacy of the posterior distribution or privatizing off-the-shelf Bayesian methods, these works generally do not come with rigorous utility guarantees beyond low-dimensional settings. In fact, even for the prototypical tasks of Gaussian mean estimation and linear regression, it was unknown how close one could get to the Bayes-optimal error with a private algorithm, even in the simplest case where the unknown parameter comes from a Gaussian prior. In this work, we give the first efficient algorithms for both of these problems that achieve mean-squared error $(1+o(1))\mathrm{OPT}$ and additionally show that both tasks exhibit an intriguing computational-statistical gap. For Bayesian mean estimation, we prove that the excess risk achieved by our method is optimal among all efficient algorithms within the low-degree framework, yet is provably worse than what is achievable by an exponential-time algorithm. For linear regression, we prove a qualitatively similar lower bound. Our algorithms draw upon the privacy-to-robustness framework of arXiv:2212.05015, but with the curious twist that to achieve private Bayes-optimal estimation, we need to design sum-of-squares-based robust estimators for inherently non-robust objects like the empirical mean and OLS estimator. Along the way we also add to the sum-of-squares toolkit a new kind of constraint based on short-flat decompositions.

翻译：贝叶斯方法位于现代数据科学的核心，为数据受限环境下的估计以及不确定性的原则性量化与传播提供了强大框架。然而，在这些方法实际部署的许多场景中，自然需要保护被审查数据个体的隐私。尽管已有众多工作试图通过探讨后验分布的内在隐私性或对现成贝叶斯方法进行隐私化处理来应对差分隐私贝叶斯估计问题，但这些工作通常无法在低维设置之外提供严格的效用保证。事实上，即使对于高斯均值估计和线性回归这类典型任务，即使未知参数服从高斯先验这一最简单情形，我们此前也未知私有算法能多接近贝叶斯最优误差。在本工作中，我们首次为这两个问题给出了高效算法，实现了均方误差$(1+o(1))\mathrm{OPT}$，并进一步证明这两个任务均展现出引人注目的计算-统计间隙。对于贝叶斯均值估计，我们证明所提方法在低度框架下是所有高效算法中最优的，但其超额风险确实劣于指数时间算法所能达到的水平。对于线性回归，我们证明了定性相似的下界。我们的算法借鉴了arXiv:2212.05015中的隐私-鲁棒性框架，但存在一个关键转折：为实现私有贝叶斯最优估计，需要针对本质上非鲁棒的对象（如经验均值和OLS估计量）设计基于平方和方法的鲁棒估计量。在此过程中，我们还基于短-平坦分解为平方和方法工具箱新增了一类约束。