In this paper, we present new high-probability PAC-Bayes bounds for different types of losses. Firstly, for losses with a bounded range, we recover a strengthened version of Catoni's bound that holds uniformly for all parameter values. This leads to new fast rate and mixed rate bounds that are interpretable and tighter than previous bounds in the literature. In particular, the fast rate bound is equivalent to the Seeger--Langford bound. Secondly, for losses with more general tail behaviors, we introduce two new parameter-free bounds: a PAC-Bayes Chernoff analogue when the loss' cumulative generating function is bounded, and a bound when the loss' second moment is bounded. These two bounds are obtained using a new technique based on a discretization of the space of possible events for the "in probability" parameter optimization problem. This technique is both simpler and more general than previous approaches optimizing over a grid on the parameters' space. Finally, we extend all previous results to anytime-valid bounds using a simple technique applicable to any existing bound.

翻译：本文针对不同类型的损失函数提出了新的高概率PAC-Bayes界。首先，对于有界范围的损失函数，我们恢复了Catoni界的一个强化版本，该版本对所有参数值一致成立。这导致了新的快速率界和混合率界，这些界比文献中先前的界更易于解释且更紧凑。特别地，快速率界等价于Seeger-Langford界。其次，对于具有更一般尾部行为的损失函数，我们引入了两个无参数界：当损失函数的累积生成函数有界时的PAC-Bayes切诺夫界类比，以及当损失函数的二阶矩有界时的界。这两个界通过一种基于对"依概率"参数优化问题可能事件空间进行离散化的新技术获得，该技术比先前在参数空间网格上优化的方法更简单且更通用。最后，我们利用一种适用于任何现有界的简单技术将所有先前结果扩展到任意时间有效的界。