In this paper we compare and contrast the behavior of the posterior predictive distribution to the risk of the maximum a posteriori estimator for the random features regression model in the overparameterized regime. We will focus on the variance of the posterior predictive distribution (Bayesian model average) and compare its asymptotics to that of the risk of the MAP estimator. In the regime where the model dimensions grow faster than any constant multiple of the number of samples, asymptotic agreement between these two quantities is governed by the phase transition in the signal-to-noise ratio. They also asymptotically agree with each other when the number of samples grow faster than any constant multiple of model dimensions. Numerical simulations illustrate finer distributional properties of the two quantities for finite dimensions. We conjecture they have Gaussian fluctuations and exhibit similar properties as found by previous authors in a Gaussian sequence model, which is of independent theoretical interest.

翻译：本文比较并对比了过参数化场景下随机特征回归模型中后验预测分布与最大后验估计器风险的行为。我们将重点分析后验预测分布（贝叶斯模型平均）的方差，并比较其与MAP估计器风险的渐近性质。当模型维度增长速度超过样本数的任意常数倍时，这两者之间的渐近一致性由信噪比的相变决定。而当样本数增长速度超过模型维度的任意常数倍时，它们在渐近意义上也相互一致。数值模拟揭示了有限维情况下两个量的精细分布特性。我们推测它们具有高斯涨落，并表现出与先前学者在高斯序列模型中发现的性质相似的特征——这一现象本身便具有独立的理论研究价值。