We present a unified framework for deriving PAC-Bayesian generalization bounds. Unlike most previous literature on this topic, our bounds are anytime-valid (i.e., time-uniform), meaning that they hold at all stopping times, not only for a fixed sample size. Our approach combines four tools in the following order: (a) nonnegative supermartingales or reverse submartingales, (b) the method of mixtures, (c) the Donsker-Varadhan formula (or other convex duality principles), and (d) Ville's inequality. Our main result is a PAC-Bayes theorem which holds for a wide class of discrete stochastic processes. We show how this result implies time-uniform versions of well-known classical PAC-Bayes bounds, such as those of Seeger, McAllester, Maurer, and Catoni, in addition to many recent bounds. We also present several novel bounds. Our framework also enables us to relax traditional assumptions; in particular, we consider nonstationary loss functions and non-i.i.d. data. In sum, we unify the derivation of past bounds and ease the search for future bounds: one may simply check if our supermartingale or submartingale conditions are met and, if so, be guaranteed a (time-uniform) PAC-Bayes bound.

翻译：我们提出一个用于推导PAC-贝叶斯泛化界限的统一框架。与以往多数相关文献不同，本框架所得界限具有任意时点有效性（即时间一致性），这意味着这些界限适用于所有停时，而非仅针对固定样本量。该方法按序整合四种工具：(a) 非负上鞅或逆下鞅，(b) 混合方法，(c) 唐斯克-瓦拉丹公式（或其他凸对偶原理），及(d) 维尔不等式。主要成果是一个适用于广泛离散随机过程类别的PAC-贝叶斯定理。我们展示该定理如何推导出经典PAC-贝叶斯界限（如西格、麦卡莱斯特、毛雷尔和卡托尼提出的界限）的时间一致版本，同时涵盖众多最新界限。我们还提出若干新型界限。该框架还能放松传统假设：具体而言，考虑了非平稳损失函数及非独立同分布数据。总之，我们统一了历史界限的推导过程，并简化了未来界限的探索路径：只需验证是否满足上鞅或下鞅条件，即可保证获得（时间一致的）PAC-贝叶斯界限。