Gaussian processes (GPs) are non-parametric probabilistic regression models that are popular due to their flexibility, data efficiency, and well-calibrated uncertainty estimates. However, standard GP models assume homoskedastic Gaussian noise, while many real-world applications are subject to non-Gaussian corruptions. Variants of GPs that are more robust to alternative noise models have been proposed, and entail significant trade-offs between accuracy and robustness, and between computational requirements and theoretical guarantees. In this work, we propose and study a GP model that achieves robustness against sparse outliers by inferring data-point-specific noise levels with a sequential selection procedure maximizing the log marginal likelihood that we refer to as relevance pursuit. We show, surprisingly, that the model can be parameterized such that the associated log marginal likelihood is strongly concave in the data-point-specific noise variances, a property rarely found in either robust regression objectives or GP marginal likelihoods. This in turn implies the weak submodularity of the corresponding subset selection problem, and thereby proves approximation guarantees for the proposed algorithm. We compare the model's performance relative to other approaches on diverse regression and Bayesian optimization tasks, including the challenging but common setting of sparse corruptions of the labels within or close to the function range.

翻译：高斯过程（GPs）是一种非参数概率回归模型，因其灵活性、数据效率以及校准良好的不确定性估计而广受欢迎。然而，标准GP模型假设同方差高斯噪声，而许多实际应用却受到非高斯噪声的干扰。已有研究提出了对替代噪声模型更具鲁棒性的GP变体，但这些方法在精度与鲁棒性之间、计算需求与理论保证之间均存在显著的权衡。本文提出并研究了一种GP模型，该模型通过一种最大化对数边缘似然的顺序选择过程（我们称之为相关性追踪）来推断数据点特定的噪声水平，从而实现对稀疏异常值的鲁棒性。令人惊讶的是，我们证明该模型可以参数化，使得相关的对数边缘似然在数据点特定噪声方差上具有强凹性——这一性质在鲁棒回归目标或GP边缘似然中均属罕见。这进而意味着相应子集选择问题具有弱子模性，从而为所提算法提供了近似保证。我们在多种回归和贝叶斯优化任务上，将该模型的性能与其他方法进行了比较，包括在函数值域内或附近对标签进行稀疏污染这一常见且具有挑战性的场景。