In this paper we compare and contrast the behavior of the posterior predictive distribution to the risk of the maximum a posteriori estimator for the random features regression model in the overparameterized regime. We will focus on the variance of the posterior predictive distribution (Bayesian model average) and compare its asymptotics to that of the risk of the MAP estimator. In the regime where the model dimensions grow faster than any constant multiple of the number of samples, asymptotic agreement between these two quantities is governed by the phase transition in the signal-to-noise ratio. They also asymptotically agree with each other when the number of samples grow faster than any constant multiple of model dimensions. Numerical simulations illustrate finer distributional properties of the two quantities for finite dimensions. We conjecture they have Gaussian fluctuations and exhibit similar properties as found by previous authors in a Gaussian sequence model, which is of independent theoretical interest.

翻译：本文比较并对比了过参数化机制下随机特征回归模型中后验预测分布与最大后验估计器风险的行为。我们将重点研究后验预测分布（贝叶斯模型平均）的方差，并将其渐近性与MAP估计器的风险进行比较。在模型维度增速超过样本数任何常数倍的情况下，这两种量的渐近一致性受信噪比相变支配。当样本数增速超过模型维度任何常数倍时，二者也呈现渐近一致性。数值模拟展示了有限维度下两种量的精细分布特性。我们猜想它们具有高斯波动，并呈现出与先前学者在高斯序列模型中所发现的类似性质，这具有独立的理论研究价值。