We derive explicit non-asymptotic PAC-Bayes generalization bounds for Gibbs posteriors, that is, data-dependent distributions over model parameters obtained by exponentially tilting a prior with the empirical risk. Unlike classical worst-case complexity bounds based on uniform laws of large numbers, which require explicit control of the model space in terms of metric entropy (integrals), our analysis yields posterior-averaged risk bounds that can be applied to overparameterized models and adapt to the data structure and the intrinsic model complexity. The bound involves a marginal-type integral over the parameter space, which we analyze using tools from singular learning theory to obtain explicit and practically meaningful characterizations of the posterior risk. Applications to low-rank matrix completion and ReLU neural network regression and classification show that the resulting bounds are analytically tractable and substantially tighter than classical complexity-based bounds. Our results highlight the potential of PAC-Bayes analysis for precise finite-sample generalization guarantees in modern overparameterized and singular models.

翻译：我们推导了吉布斯后验的显式非渐近PAC-Bayes泛化界，即通过对先验进行经验风险指数倾斜而得到的模型参数上的数据相关分布。与基于一致大数定律的经典最坏情况复杂度界（需要借助度量熵（积分）显式控制模型空间）不同，我们的分析给出了适用于过参数化模型、能自适应数据结构和模型内在复杂性的后验平均风险界。该界涉及参数空间上的边际型积分，我们利用奇异学习理论中的工具对其进行分析，从而获得关于后验风险的显式且具有实际意义的刻画。在低秩矩阵补全、ReLU神经网络回归与分类中的应用表明，所得界的解析可处理性显著优于经典的基于复杂度的方法，且实质上更紧致。我们的结果凸显了PAC-Bayes分析在现代过参数化与奇异模型中提供精确有限样本泛化保证的潜力。