Gaussian processes (GPs) are powerful and widely used probabilistic regression models, but their effectiveness in practice is often limited by the choice of kernel function. This kernel function is typically handcrafted from a small set of standard functions, a process that requires expert knowledge, results in limited adaptivity to data, and imposes strong assumptions on the hypothesis space. We study Empirical GPs, a principled framework for constructing flexible, data-driven GP priors that overcome these limitations. Rather than relying on standard parametric kernels, we estimate the mean and covariance functions empirically from a corpus of historical observations, enabling the prior to reflect rich, non-trivial covariance structures present in the data. Theoretically, we show that the resulting model converges to the GP that is closest (in KL-divergence sense) to the real data generating process. Practically, we formulate the problem of learning the GP prior from independent datasets as likelihood estimation and derive an Expectation-Maximization algorithm with closed-form updates, allowing the model handle heterogeneous observation locations across datasets. We demonstrate that Empirical GPs achieve competitive performance on learning curve extrapolation and time series forecasting benchmarks.

翻译：高斯过程（GPs）是强大且广泛使用的概率回归模型，但其在实际应用中的效果常受限于核函数的选择。该核函数通常是从一小组标准函数中手工设计的，这一过程需要专业知识，导致对数据的适应性有限，并对假设空间施加了强假设。我们研究了经验高斯过程，这是一个构建灵活、数据驱动的GP先验的原则性框架，旨在克服这些限制。我们不依赖标准的参数化核函数，而是从一个历史观测数据集中经验性地估计均值和协方差函数，从而使先验能够反映数据中存在的丰富且非平凡的协方差结构。理论上，我们证明了所得模型收敛于（在KL散度意义上）最接近真实数据生成过程的高斯过程。实践上，我们将从独立数据集中学习GP先验的问题表述为似然估计，并推导出一种具有闭式更新的期望最大化算法，使模型能够处理跨数据集的异构观测位置。我们证明，经验高斯过程在学习曲线外推和时间序列预测基准测试中取得了有竞争力的性能。