Approximate inference in Gaussian process (GP) models with non-conjugate likelihoods gets entangled with the learning of the model hyperparameters. We improve hyperparameter learning in GP models and focus on the interplay between variational inference (VI) and the learning target. While VI's lower bound to the marginal likelihood is a suitable objective for inferring the approximate posterior, we show that a direct approximation of the marginal likelihood as in Expectation Propagation (EP) is a better learning objective for hyperparameter optimization. We design a hybrid training procedure to bring the best of both worlds: it leverages conjugate-computation VI for inference and uses an EP-like marginal likelihood approximation for hyperparameter learning. We compare VI, EP, Laplace approximation, and our proposed training procedure and empirically demonstrate the effectiveness of our proposal across a wide range of data sets.

翻译：在高斯过程（GP）模型中，非共轭似然下的近似推断与模型超参数的学习相互纠缠。我们针对GP模型改进了超参数学习，并重点关注变分推断（VI）与学习目标之间的相互作用。虽然VI对边际似然的下界是推断近似后验的合适目标，但我们证明，与期望传播（EP）类似的边际似然直接近似是更优的超参数优化学习目标。我们设计了一种混合训练过程以融合两者优势：它利用共轭计算VI进行推断，并采用类似EP的边际似然近似进行超参数学习。我们比较了VI、EP、拉普拉斯近似及我们提出的训练过程，并通过广泛数据集实验证明了该方法的有效性。