Prior beliefs about the latent function to shape inductive biases can be incorporated into a Gaussian Process (GP) via the kernel. However, beyond kernel choices, the decision-making process of GP models remains poorly understood. In this work, we contribute an analysis of the loss landscape for GP models using methods from physics. We demonstrate $\nu$-continuity for Matern kernels and outline aspects of catastrophe theory at critical points in the loss landscape. By directly including $\nu$ in the hyperparameter optimisation for Matern kernels, we find that typical values of $\nu$ are far from optimal in terms of performance, yet prevail in the literature due to the increased computational speed. We also provide an a priori method for evaluating the effect of GP ensembles and discuss various voting approaches based on physical properties of the loss landscape. The utility of these approaches is demonstrated for various synthetic and real datasets. Our findings provide an enhanced understanding of the decision-making process behind GPs and offer practical guidance for improving their performance and interpretability in a range of applications.

翻译：关于潜在函数的先验信念可通过核函数融入高斯过程（GP）以塑造归纳偏差。然而，除核函数选择外，GP模型的决策机制仍未被充分理解。本文借助物理学方法，对GP模型的损失景观进行了分析。我们论证了Matern核的$ \nu $-连续性，并概述了损失景观临界点处突变理论的相关特征。通过将$ \nu $直接纳入Matern核的超参数优化过程，我们发现典型$ \nu $值在性能上远非最优，却因计算速度优势而在文献中普遍存在。此外，我们提出了一种先验方法用于评估GP集成效果，并基于损失景观的物理特性讨论了多种投票机制。在合成与真实数据集上的实验验证了这些方法的有效性。本研究深化了对GP决策过程的理解，并为提升其在实际应用中的性能与可解释性提供了实践指导。