Approximate Bayesian inference typically revolves around computing the posterior parameter distribution. In practice, however, the main object of interest is often a model's predictions rather than its parameters. In this work, we propose to bypass the parameter posterior and focus directly on approximating the posterior predictive distribution. We achieve this by drawing inspiration from self-training within self-supervised and semi-supervised learning. Essentially, we quantify a Bayesian model's predictive uncertainty by refitting on self-predicted data. The idea is strikingly simple: If a model assigns high likelihood to self-predicted data, these predictions are of low uncertainty, and vice versa. This yields a deterministic, sampling-free approximation of the posterior predictive. The modular structure of our Self-Supervised Laplace Approximation (SSLA) further allows us to plug in different prior specifications, enabling classical Bayesian sensitivity (w.r.t. prior choice) analysis. In order to bypass expensive refitting, we further introduce an approximate version of SSLA, called ASSLA. We study (A)SSLA both theoretically and empirically in regression models ranging from Bayesian linear models to Bayesian neural networks. Across a wide array of regression tasks with simulated and real-world datasets, our methods outperform classical Laplace approximations in predictive calibration while remaining computationally efficient.

翻译：近似贝叶斯推断通常围绕计算后验参数分布展开。然而在实践中，人们关注的核心往往是模型的预测结果而非其参数。本文提出绕过参数后验分布，直接聚焦于近似后验预测分布。我们受自监督与半监督学习中自训练的启发，通过将模型在自预测数据上重新拟合来量化贝叶斯模型的预测不确定性。其核心理念极为简洁：若模型对自预测数据赋予高似然值，则这些预测的不确定性较低，反之亦然。这便产生了后验预测的一种确定性、无采样的近似方法。我们提出的自我监督拉普拉斯近似（SSLA）具有模块化结构，可嵌入不同的先验设定，实现经典贝叶斯灵敏度（相对于先验选择）分析。为规避昂贵的重新拟合过程，我们进一步引入SSLA的近似版本ASSLA。本文从理论和实证两个层面对（A）SSLA进行了研究，涵盖从贝叶斯线性模型到贝叶斯神经网络的回归模型。在基于模拟数据和真实数据集的大量回归任务中，我们的方法在保持计算高效性的同时，其预测校准能力优于传统拉普拉斯近似方法。