The sample complexity of estimating or maximising an unknown function in a reproducing kernel Hilbert space is known to be linked to both the effective dimension and the information gain associated with the kernel. While the information gain has an attractive information-theoretic interpretation, the effective dimension typically results in better rates. We introduce a new quantity called the relative information gain, which measures the sensitivity of the information gain with respect to the observation noise. We show that the relative information gain smoothly interpolates between the effective dimension and the information gain, and that the relative information gain has the same growth rate as the effective dimension. In the second half of the paper, we prove a new PAC-Bayesian excess risk bound for Gaussian process regression. The relative information gain arises naturally from the complexity term in this PAC-Bayesian bound. We prove bounds on the relative information gain that depend on the spectral properties of the kernel. When these upper bounds are combined with our excess risk bound, we obtain minimax-optimal rates of convergence.

翻译：在再生核希尔伯特空间中估计或最大化未知函数的样本复杂度，已知与核的有效维度和信息增益均有关联。尽管信息增益具有吸引人的信息论解释，但有效维度通常能带来更优的收敛速率。本文引入了一个称为相对信息增益的新度量，它衡量信息增益对观测噪声的敏感性。我们证明相对信息增益在有效维度和信息增益之间实现了平滑插值，且其增长率与有效维度相同。在论文后半部分，我们为高斯过程回归证明了一个新的PAC贝叶斯超额风险界。相对信息增益自然地出现在该PAC贝叶斯界的复杂度项中。我们证明了依赖于核谱性质的相对信息增益上界。当这些上界与我们的超额风险界结合时，我们获得了极小化极大最优的收敛速率。