Approximate Bayesian inference typically revolves around computing the posterior parameter distribution. In practice, however, the main object of interest is often a model's predictions rather than its parameters. In this work, we propose to bypass the parameter posterior and focus directly on approximating the posterior predictive distribution. We achieve this by drawing inspiration from self-training within self-supervised and semi-supervised learning. Essentially, we quantify a Bayesian model's predictive uncertainty by refitting on self-predicted data. The idea is strikingly simple: If a model assigns high likelihood to self-predicted data, these predictions are of low uncertainty, and vice versa. This yields a deterministic, sampling-free approximation of the posterior predictive. The modular structure of our Self-Supervised Laplace Approximation (SSLA) further allows us to plug in different prior specifications, enabling classical Bayesian sensitivity (w.r.t. prior choice) analysis. In order to bypass expensive refitting, we further introduce an approximate version of SSLA, called ASSLA. We study (A)SSLA both theoretically and empirically in regression models ranging from Bayesian linear models to Bayesian neural networks. Across a wide array of regression tasks with simulated and real-world datasets, our methods outperform classical Laplace approximations in predictive calibration while remaining computationally efficient.

翻译：近似贝叶斯推断通常围绕计算后验参数分布展开。然而在实践中，我们关注的核心对象往往是模型的预测结果而非其参数。本研究提出绕过参数后验，直接聚焦于近似后验预测分布。我们通过借鉴自监督和半监督学习中的自训练思想来实现这一目标：本质上，通过重新拟合自预测数据来量化贝叶斯模型的预测不确定性。该思想极为简洁——若模型对自预测数据赋予高似然，则表明这些预测具有低不确定性，反之亦然。这提供了后验预测的一种确定性、免采样的近似方法。我们的自监督拉普拉斯近似（SSLA）的模块化结构允许灵活嵌入不同先验设定，进而实现经典的贝叶斯敏感性（关于先验选择）分析。为规避昂贵的重拟合过程，我们还引入了SSLA的近似版本ASSLA。我们从理论和实证两个维度，在从贝叶斯线性模型到贝叶斯神经网络的回归模型中研究（A）SSLA。在涉及仿真与真实数据集的广泛回归任务中，我们的方法在保持计算高效性的同时，在预测校准方面优于经典拉普拉斯近似。