Combining Gaussian processes with the expressive power of deep neural networks is commonly done nowadays through deep kernel learning (DKL). Unfortunately, due to the kernel optimization process, this often results in losing their Bayesian benefits. In this study, we present a novel approach for learning deep kernels by utilizing infinite-width neural networks. We propose to use the Neural Network Gaussian Process (NNGP) model as a guide to the DKL model in the optimization process. Our approach harnesses the reliable uncertainty estimation of the NNGPs to adapt the DKL target confidence when it encounters novel data points. As a result, we get the best of both worlds, we leverage the Bayesian behavior of the NNGP, namely its robustness to overfitting, and accurate uncertainty estimation, while maintaining the generalization abilities, scalability, and flexibility of deep kernels. Empirically, we show on multiple benchmark datasets of varying sizes and dimensionality, that our method is robust to overfitting, has good predictive performance, and provides reliable uncertainty estimations.

翻译：将高斯过程与深度神经网络的强大表达能力相结合，目前通常通过深度核学习（DKL）实现。然而，由于核优化过程，这常常导致其贝叶斯优势的丧失。在本研究中，我们提出了一种利用无限宽神经网络学习深度核的新方法。我们建议在优化过程中使用神经网络高斯过程（NNGP）模型作为DKL模型的引导。该方法利用NNGP可靠的置信度估计，在遇到新的数据点时调整DKL的目标置信度。由此，我们实现了两全其美：既利用了NNGP的贝叶斯特性，即其对过拟合的鲁棒性和准确的置信度估计，又保持了深度核的泛化能力、可扩展性和灵活性。通过多个不同规模和维度的基准数据集实验，我们证明了该方法对过拟合具有鲁棒性，具有良好的预测性能，并能提供可靠的置信度估计。