Prior beliefs about the latent function to shape inductive biases can be incorporated into a Gaussian Process (GP) via the kernel. However, beyond kernel choices, the decision-making process of GP models remains poorly understood. In this work, we contribute an analysis of the loss landscape for GP models using methods from physics. We demonstrate $\nu$-continuity for Matern kernels and outline aspects of catastrophe theory at critical points in the loss landscape. By directly including $\nu$ in the hyperparameter optimisation for Matern kernels, we find that typical values of $\nu$ are far from optimal in terms of performance, yet prevail in the literature due to the increased computational speed. We also provide an a priori method for evaluating the effect of GP ensembles and discuss various voting approaches based on physical properties of the loss landscape. The utility of these approaches is demonstrated for various synthetic and real datasets. Our findings provide an enhanced understanding of the decision-making process behind GPs and offer practical guidance for improving their performance and interpretability in a range of applications.

翻译：关于潜在函数的先验信念可以通过核函数纳入高斯过程，以塑造归纳偏置。然而，除核函数选择外，高斯过程模型的决策机制仍缺乏深入理解。本研究利用物理学方法对高斯过程模型的损失景观展开分析。我们证明了马特恩核的ν-连续性，并阐述了损失景观临界点处的突变理论特征。通过将ν直接纳入马特恩核的超参数优化，发现典型ν值在性能上远非最优，却因计算效率提升而在文献中普遍存在。此外，我们提出了一种先验方法来评估高斯过程集成的效果，并基于损失景观的物理性质讨论了多种投票策略。通过在合成及真实数据集上的实验验证了这些方法的有效性。研究成果深化了对高斯过程决策机制的理解，为提升其在实际应用中的性能与可解释性提供了实践指导。