Deep Gaussian processes (DGPs) provide a robust paradigm for Bayesian deep learning. In DGPs, a set of sparse integration locations called inducing points are selected to approximate the posterior distribution of the model. This is done to reduce computational complexity and improve model efficiency. However, inferring the posterior distribution of inducing points is not straightforward. Traditional variational inference approaches to posterior approximation often lead to significant bias. To address this issue, we propose an alternative method called Denoising Diffusion Variational Inference (DDVI) that uses a denoising diffusion stochastic differential equation (SDE) to generate posterior samples of inducing variables. We rely on score matching methods for denoising diffusion model to approximate score functions with a neural network. Furthermore, by combining classical mathematical theory of SDEs with the minimization of KL divergence between the approximate and true processes, we propose a novel explicit variational lower bound for the marginal likelihood function of DGP. Through experiments on various datasets and comparisons with baseline methods, we empirically demonstrate the effectiveness of DDVI for posterior inference of inducing points for DGP models.

翻译：深度高斯过程（DGPs）为贝叶斯深度学习提供了一个稳健的范式。在DGPs中，一组被称为诱导点的稀疏积分位置被选取用于近似模型的后验分布。这样做是为了降低计算复杂度并提高模型效率。然而，推断诱导点的后验分布并非易事。传统的用于后验近似的变分推断方法常常导致显著的偏差。为了解决这个问题，我们提出了一种名为去噪扩散变分推断（DDVI）的替代方法，该方法使用去噪扩散随机微分方程（SDE）来生成诱导变量的后验样本。我们依赖去噪扩散模型的分数匹配方法，通过神经网络来近似分数函数。此外，通过将经典的SDE数学理论与近似过程和真实过程之间KL散度的最小化相结合，我们为DGP的边缘似然函数提出了一种新颖的显式变分下界。通过在多个数据集上的实验以及与基线方法的比较，我们实证证明了DDVI在DGP模型诱导点后验推断方面的有效性。